ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering

Paper content:
Answering questions that require multi-hop reasoning at web-scale necessitates retrieving multiple evidence documents, one of which often has little lexical or semantic relationship to the question.
This paper introduces a new graph-based recurrent retrieval approach that learns to retrieve reasoning paths over the Wikipedia graph to answer multi-hop open-domain questions.
Our retriever model trains a recurrent neural network that learns to sequentially retrieve evidence paragraphs in the reasoning path by conditioning on the previously retrieved documents.
Our reader model ranks the reasoning paths and extracts the answer span included in the best reasoning path.
Experimental results show state-of-the-art results in three open-domain QA datasets, showcasing the effectiveness and robustness of our method.
Notably, our method achieves significant improvement in HotpotQA, outperforming the previous best model by more than 14 points.

assistant: text: Graph-based recurrent retriever that learns to retrieve reasoning paths over Wikipedia Graph outperforms the most recent state of the art on HotpotQA by more than 14 points.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Making question answering more robust through relevant context selection

Paper content:
Existing neural question answering (QA) models are required to reason over and draw complicated inferences from a long context for most large-scale QA datasets.
However, if we view QA as a combined retrieval and reasoning task, we can assume the existence of a minimal context which is necessary and sufficient to answer a given question.
Recent work has shown that a sentence selector module that selects a shorter context and feeds it to the downstream QA model achieves performance comparable to a QA model trained on full context, while also being more interpretable.
Recent work has also shown that most state-of-the-art QA models break when adversarially generated sentences are appended to the context.
While humans are immune to such distractor sentences, QA models get easily misled into selecting answers from these sentences.
We hypothesize that the sentence selector module can filter out extraneous context, thereby allowing the downstream QA model to focus and reason over the parts of the context that are relevant to the question.
In this paper, we show that the sentence selector itself is susceptible to adversarial inputs.
However, we demonstrate that a pipeline consisting of a sentence selector module followed by the QA model can be made more robust to adversarial attacks in comparison to a QA model trained on full context.
Thus, we provide evidence towards a modular approach for question answering that is more robust and interpretable.

assistant: text: A modular approach consisting of a sentence selector module followed by the QA model can be made more robust to adversarial attacks in comparison to a QA model trained on full context.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Hallucinative Topological Memory for Zero-Shot Visual Planning

Paper content:
In visual planning (VP), an agent learns to plan goal-directed behavior from observations of a dynamical system obtained offline, e.g., images obtained from self-supervised robot interaction.
VP algorithms essentially combine data-driven perception and planning, and are important for robotic manipulation and navigation domains, among others.
A recent and promising approach to VP is the semi-parametric topological memory (SPTM) method, where image samples are treated as nodes in a graph, and the connectivity in the graph is learned using deep image classification.
Thus, the learned graph represents the topological connectivity of the data, and planning can be performed using conventional graph search methods.
However, training SPTM necessitates a suitable loss function for the connectivity classifier, which requires non-trivial manual tuning.
More importantly, SPTM is constricted in its ability to generalize to changes in the domain, as its graph is constructed from direct observations and thus requires collecting new samples for planning.
In this paper, we propose Hallucinative Topological Memory (HTM), which overcomes these shortcomings.
In HTM, instead of training a discriminative classifier we train an energy function using contrastive predictive coding.
In addition, we learn a conditional VAE model that generates samples given a context image of the domain, and use these hallucinated samples for building the connectivity graph, allowing for zero-shot generalization to domain changes.
In simulated domains, HTM outperforms conventional SPTM and visual foresight methods in terms of both plan quality and success in long-horizon planning.

assistant: text: We propose Hallucinative Topological Memory (HTM), a visual planning algorithm that can perform zero-shot long horizon planning in new environments.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Physiological Signal Embeddings (PHASE) via Interpretable Stacked Models

Paper content:
In health, machine learning is increasingly common, yet neural network embedding (representation) learning is arguably under-utilized for physiological signals.
This inadequacy stands out in stark contrast to more traditional computer science domains, such as computer vision (CV), and natural language processing (NLP).
For physiological signals, learning feature embeddings is a natural solution to data insufficiency caused by patient privacy concerns -- rather than share data, researchers may share informative embedding models (i.e., representation models), which map patient data to an output embedding.
Here, we present the PHASE (PHysiologicAl Signal Embeddings) framework, which consists of three components: i) learning neural network embeddings of physiological signals, ii) predicting outcomes based on the learned embedding, and iii) interpreting the prediction results by estimating feature attributions in the "stacked" models (i.e., feature embedding model followed by prediction model).
PHASE is novel in three ways: 1) To our knowledge, PHASE is the first instance of transferal of neural networks to create physiological signal embeddings.
2) We present a tractable method to obtain feature attributions through stacked models.
We prove that our stacked model attributions can approximate Shapley values -- attributions known to have desirable properties -- for arbitrary sets of models.
3) PHASE was extensively tested in a cross-hospital setting including publicly available data.
In our experiments, we show that PHASE significantly outperforms alternative embeddings -- such as raw, exponential moving average/variance, and autoencoder -- currently in use.
Furthermore, we provide evidence that transferring neural network embedding/representation learners between distinct hospitals still yields performant embeddings and offer recommendations when transference is ineffective.

assistant: text: Physiological signal embeddings for prediction performance and hospital transference with a general Shapley value interpretability method for stacked models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Ada-Boundary: Accelerating the DNN Training via Adaptive Boundary Batch Selection

Paper content:
Neural networks can converge faster with help from a smarter batch selection strategy.
In this regard, we propose Ada-Boundary, a novel adaptive-batch selection algorithm that constructs an effective mini-batch according to the learning progress of the model.Our key idea is to present confusing samples what the true label is.
Thus, the samples near the current decision boundary are considered as the most effective to expedite convergence.
Taking advantage of our design, Ada-Boundary maintains its dominance in various degrees of training difficulty.
We demonstrate the advantage of Ada-Boundary by extensive experiments using two convolutional neural networks for three benchmark data sets.
The experiment results show that Ada-Boundary improves the training time by up to 31.7% compared with the state-of-the-art strategy and by up to 33.5% compared with the baseline strategy.

assistant: text: We suggest a smart batch selection technique called Ada-Boundary.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Twitter User Sentiments on Climate Change with Limited Labeled Data

Paper content:
While it is well-documented that climate change accepters and deniers have become increasingly polarized in the United States over time, there has been no large-scale examination of whether these individuals are prone to changing their opinions as a result of natural external occurrences.
On the sub-population of Twitter users, we examine whether climate change sentiment changes in response to five separate natural disasters occurring in the U.S. in 2018.
We begin by showing that tweets can be classified with over 75% accuracy as either accepting or denying climate change when using our methodology to compensate for limited labelled data; results are robust across several machine learning models and yield geographic-level results in line with prior research.
We then apply RNNs to conduct a cohort-level analysis showing that the 2018 hurricanes yielded a statistically significant increase in average tweet sentiment affirming climate change.
However, this effect does not hold for the 2018 blizzard and wildfires studied, implying that Twitter users' opinions on climate change are fairly ingrained on this subset of natural disasters.

assistant: text: We train RNNs on famous Twitter users to determine whether the general Twitter population is more likely to believe in climate change after a natural disaster.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Do Language Models Have Common Sense?

Paper content:
It has been argued that current machine learning models do not have commonsense, and therefore must be hard-coded with prior knowledge (Marcus, 2018).
Here we show surprising evidence that language models can already learn to capture certain common sense knowledge.
Our key observation is that a language model can compute the probability of any statement, and this probability can be used to evaluate the truthfulness of that statement.
On the Winograd Schema Challenge (Levesque et al., 2011), language models are 11% higher in accuracy than previous state-of-the-art supervised methods.
Language models can also be fine-tuned for the task of Mining Commonsense Knowledge on ConceptNet to achieve an F1 score of 0.912 and 0.824, outperforming previous best results (Jastrzebskiet al., 2018).
Further analysis demonstrates that language models can discover unique features of Winograd Schema contexts that decide the correct answers without explicit supervision.

assistant: text: We present evidence that LMs do capture common sense with state-of-the-art results on both Winograd Schema Challenge and Commonsense Knowledge Mining.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Action Semantics Network: Considering the Effects of Actions in Multiagent Systems

Paper content:
In multiagent systems (MASs), each agent makes individual decisions but all of them contribute globally to the system evolution.
Learning in MASs is difficult since each agent's selection of actions must take place in the presence of other co-learning agents.
Moreover, the environmental stochasticity and uncertainties increase exponentially with the increase in the number of agents.
Previous works borrow various multiagent coordination mechanisms into deep learning architecture to facilitate multiagent coordination.
However, none of them explicitly consider action semantics between agents that different actions have different influences on other agents.
In this paper, we propose a novel network architecture, named Action Semantics Network (ASN), that explicitly represents such action semantics between agents.
ASN characterizes different actions' influence on other agents using neural networks based on the action semantics between them.
ASN can be easily combined with existing deep reinforcement learning (DRL) algorithms to boost their performance.
Experimental results on StarCraft II micromanagement and Neural MMO show ASN significantly improves the performance of state-of-the-art DRL approaches compared with several network architectures.

assistant: text: Our proposed ASN characterizes different actions' influence on other agents using neural networks based on the action semantics between them.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efﬁcient Neural Network Compression via Transfer Learning for Industrial Optical Inspection

Paper content:
In this paper, we investigate learning the deep neural networks for automated optical inspection in industrial manufacturing.
Our preliminary result has shown the stunning performance improvement by transfer learning from the completely dissimilar source domain: ImageNet.
Further study for demystifying this improvement shows that the transfer learning produces a highly compressible network, which was not the case for the network learned from scratch.
The experimental result shows that there is a negligible accuracy drop in the network learned by transfer learning until it is compressed to 1/128 reduction of the number of convolution ﬁlters.
This result is contrary to the compression without transfer learning which loses more than 5% accuracy at the same compression rate.

assistant: text: We experimentally show that transfer learning makes sparse features in the network and thereby produces a more compressible network.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On Symmetry and Initialization for Neural Networks

Paper content:
This work provides an additional step in the theoretical understanding of neural networks.
We consider neural networks with one hidden layer and show that when learning symmetric functions, one can choose initial conditions so that standard SGD training efficiently produces generalization guarantees.
We empirically verify this and show that this does not hold when the initial conditions are chosen at random.
The proof of convergence investigates the interaction between the two layers of the network.
Our results highlight the importance of using symmetry in the design of neural networks.

assistant: text: When initialized properly, neural networks can learn the simple class of symmetric functions; when initialized randomly, they fail.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: What Information Does a ResNet Compress?

Paper content:
The information bottleneck principle (Shwartz-Ziv & Tishby, 2017) suggests that SGD-based training of deep neural networks results in optimally compressed hidden layers, from an information theoretic perspective.
However, this claim was established on toy data.
The goal of the work we present here is to test these claims in a realistic setting using a larger and deeper convolutional architecture, a ResNet model.
We trained PixelCNN++ models as inverse representation decoders to measure the mutual information between hidden layers of a ResNet and input image data, when trained for (1) classification and (2) autoencoding.
We find that two stages of learning happen for both training regimes, and that compression does occur, even for an autoencoder.
Sampling images by conditioning on hidden layers’ activations offers an intuitive visualisation to understand what a ResNets learns to forget.
The ResNet architecture enables very deep CNNs.
We show that learning representations using a ResNet results in information compression in hidden layers.
We set out in this research to test some of the claims by Shwartz-Ziv & Tishby (2017) regarding the information bottleneck principle applied to deep learning.
By defining a lower bound on the MI and 'decoder' models to compute the MI during classifier and autoencoder training regimes, we explored the notion of compression for generalisation in the context of realistic images and a modern architecture choice.For both classification and autoencoding we observed two stages of learning, characterised by: (1) an initial and relatively short-lived increase and (2) a longer decrease in MI between hidden layers and input training data.
Although we cannot confirm the mechanism responsible for compression (stochastic relaxation, for example), we gave an intuitive glimpse into what quality/type of information is kept and discarded as ResNets learn.
PixelCNN++ models were used to estimate the MI between hidden layers (of the models under scrutiny) and input data; images were generated conditioned on hidden layers to illustrate the fitting and compression of data in a visual and intuitive fashion.The experimental procedure we developed for this research enables visualising class invariances throughout training.
In particular, we see that when a ResNet is maximally (subject to model constraints) compressing information in its hidden layers, the class-irrelevant features of the input images are discarded: conditionally generated samples vary more while retaining information relevant to classification.
This result has been shown in theory and for toy examples, but never illustrated to the degree that we do here.

assistant: text: The Information Bottleneck Principle applied to ResNets, using PixelCNN++ models to decode mutual information and conditionally generate images for information illustration
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: EXPLOITING SEMANTIC COHERENCE TO IMPROVE PREDICTION IN SATELLITE SCENE IMAGE ANALYSIS: APPLICATION TO DISEASE DENSITY ESTIMATION

Paper content:
High intra-class diversity and inter-class similarity is a characteristic of remote sensing scene image data sets currently posing significant difficulty for deep learning algorithms on classification tasks.
To improve accuracy, post-classification
methods have been proposed for smoothing results of model predictions.
However, those approaches require an additional neural network to perform the smoothing operation, which adds overhead to the task.
We propose an approach that involves learning deep features directly over neighboring scene images without requiring use of a cleanup model.
Our approach utilizes a siamese network to improve the discriminative power of convolutional neural networks on a pair
of neighboring scene images.
It then exploits semantic coherence between this pair to enrich the feature vector of the image for which we want to predict a label.
Empirical results show that this approach provides a viable alternative to existing methods.
For example, our model improved prediction accuracy by 1 percentage point and dropped the mean squared error value by 0.02 over the baseline, on a disease density estimation task.
These performance gains are comparable with results from existing post-classification methods, moreover without implementation overheads.

assistant: text: Approach for improving prediction accuracy by learning deep features over neighboring scene images in satellite scene image analysis.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Unsupervised Discovery of Dynamic Neural Circuits

Paper content:
What can we learn about the functional organization of cortical microcircuits from large-scale recordings of neural activity?
To obtain an explicit and interpretable model of time-dependent functional connections between neurons and to establish the dynamics of the cortical information flow, we develop 'dynamic neural relational inference' (dNRI).
We study  both synthetic and real-world neural spiking data and demonstrate that the developed method is able to uncover the dynamic relations between neurons more reliably than existing baselines.

assistant: text: We develop 'dynamic neural relational inference', a variational autoencoder model that can explicitly and interpretably represent the hidden dynamic relations between neurons.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Implicit λ-Jeffreys Autoencoders: Taking the Best of Both Worlds

Paper content:
We propose a new form of an autoencoding model which incorporates the best properties of variational autoencoders (VAE) and generative adversarial networks (GAN).
It is known that GAN can produce very realistic samples while VAE does not suffer from mode collapsing problem.
Our model optimizes λ-Jeffreys divergence between the model distribution and the true data distribution.
We show that it takes the best properties of VAE and GAN objectives.
It consists of two parts.
One of these parts can be optimized by using the standard adversarial training, and the second one is the very objective of the VAE model.
However, the straightforward way of substituting the VAE loss does not work well if we use an explicit likelihood such as Gaussian or Laplace which have limited flexibility in high dimensions and are unnatural for modelling images in the space of pixels.
To tackle this problem we propose a novel approach to train the VAE model with an implicit likelihood by an adversarially trained discriminator.
In an extensive set of experiments on CIFAR-10 and TinyImagent datasets, we show that our model achieves the state-of-the-art generation and reconstruction quality and demonstrate how we can balance between mode-seeking and mode-covering behaviour of our model by adjusting the weight λ in our objective.

assistant: text: We propose a new form of an autoencoding model which incorporates the best properties of variational autoencoders (VAE) and generative adversarial networks (GAN)
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Measuring Compositionality in Representation Learning

Paper content:
Many machine learning algorithms represent input data with vector embeddings or discrete codes.
When inputs exhibit compositional structure (e.g. objects built from parts or procedures from subroutines), it is natural to ask whether this compositional structure is reflected in the the inputs’ learned representations.
While the assessment of compositionality in languages has received significant attention in linguistics and adjacent fields, the machine learning literature lacks general-purpose tools for producing graded measurements of compositional structure in more general (e.g. vector-valued) representation spaces.
We describe a procedure for evaluating compositionality by measuring how well the true representation-producing model can be approximated by a model that explicitly composes a collection of inferred representational primitives.
We use the procedure to provide formal and empirical characterizations of compositional structure in a variety of settings, exploring the relationship between compositionality and learning dynamics, human judgments, representational similarity, and generalization.

assistant: text: This paper proposes a simple procedure for evaluating compositional structure in learned representations, and uses the procedure to explore the role of compositionality in four learning problems.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Reinforcement Learning via Replica Stacking of Quantum Measurements for the Training of Quantum Boltzmann Machines

Paper content:
Recent theoretical and experimental results suggest the possibility of using current and near-future quantum hardware in challenging sampling tasks.
In this paper, we introduce free-energy-based reinforcement learning (FERL) as an application of quantum hardware.
We propose a method for processing a quantum annealer’s measured qubit spin configurations in approximating the free energy of a quantum Boltzmann machine (QBM).
We then apply this method to perform reinforcement learning on the grid-world problem using the D-Wave 2000Q quantum annealer.
The experimental results show that our technique is a promising method for harnessing the power of quantum sampling in reinforcement learning tasks.

assistant: text: We train Quantum Boltzmann Machines using a replica stacking method and a quantum annealer to perform a reinforcement learning task.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Differentiable Hebbian Consolidation for Continual Learning

Paper content:
Continual learning is the problem of sequentially learning new tasks or knowledge while protecting previously acquired knowledge.
However, catastrophic forgetting poses a grand challenge for neural networks performing such learning process.
Thus, neural networks that are deployed in the real world often struggle in scenarios where the data distribution is non-stationary (concept drift), imbalanced, or not always fully available, i.e., rare edge cases.
We propose a Differentiable Hebbian Consolidation model which is composed of a Differentiable Hebbian Plasticity (DHP) Softmax layer that adds a rapid learning plastic component (compressed episodic memory) to the fixed (slow changing) parameters of the softmax output layer; enabling learned representations to be retained for a longer timescale.
We demonstrate the flexibility of our method by integrating well-known task-specific synaptic consolidation methods to penalize changes in the slow weights that are important for each target task.
We evaluate our approach on the Permuted MNIST, Split MNIST and Vision Datasets Mixture benchmarks, and introduce an imbalanced variant of Permuted MNIST --- a dataset that combines the challenges of class imbalance and concept drift.
Our proposed model requires no additional hyperparameters and outperforms comparable baselines by reducing forgetting.

assistant: text: Hebbian plastic weights can behave as a compressed episodic memory storage in neural networks and with the combination of task-specific synaptic consolidation can improve the ability to alleviate catastrophic forgetting in continual learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards Quantum Inspired Convolution Networks

Paper content:
Deep Convolution Neural Networks (CNNs), rooted by the pioneer work of  \cite{Hinton1986,LeCun1985,Alex2012}, and summarized in \cite{LeCunBengioHinton2015},   have been shown to be very useful in a variety of fields.
The  state-of-the art CNN machines such as image rest net \cite{He_2016_CVPR} are described by real value inputs and kernel convolutions followed by the local and  non-linear rectified linear outputs.
Understanding the role of these layers, the accuracy and limitations of them,  as well as making them more efficient (fewer parameters)  are all ongoing research questions.
Inspired in quantum theory, we propose the use of complex value kernel functions, followed by the local non-linear  absolute (modulus) operator square.
We argue that an advantage of quantum inspired complex kernels is robustness to realistic unpredictable scenarios (such as clutter noise,  data deformations).
We study a concrete problem of shape detection and show that when multiple overlapping shapes are deformed and/or clutter noise is added, a convolution layer with quantum inspired complex kernels outperforms the statistical/classical kernel counterpart and a "Bayesian shape estimator" .
The superior performance is due to the quantum phenomena of interference, not present in classical CNNs.

assistant: text: A quantum inspired kernel for convolution network, exhibiting interference phenomena,  can be very useful (and compared it with real value  counterpart).
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Decoupling Gating from Linearity

Paper content:
The gap between the empirical success of deep learning and the lack of strong theoretical guarantees calls for studying simpler models.
By observing that a ReLU neuron is a product of a linear function with a gate (the latter determines whether the neuron is active or not), where both share a jointly trained weight vector, we propose to decouple the two.
We introduce GaLU networks — networks in which each neuron is a product of a Linear Unit, defined by a weight vector which is being trained, with a Gate, defined by a different weight vector which is not being trained.
Generally speaking, given a base model and a simpler version of it, the two parameters that determine the quality of the simpler version are whether its practical performance is close enough to the base model and whether it is easier to analyze it theoretically.
We show that GaLU networks perform similarly to ReLU networks on standard datasets and we initiate a study of their theoretical properties, demonstrating that they are indeed easier to analyze.
We believe that further research of GaLU networks may be fruitful for the development of a theory of deep learning.

assistant: text: We propose Gated Linear Unit networks — a model that performs similarly to ReLU networks on real data while being much easier to analyze theoretically.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Diversely Stale Parameters for Efficient Training of Deep Convolutional Networks

Paper content:
The backpropagation algorithm is the most popular algorithm training neural networks nowadays.
However, it suffers from the forward locking, backward locking and update locking problems, especially when a neural network is so large that its layers are distributed across multiple devices.
Existing solutions either can only handle one locking problem or lead to severe accuracy loss or memory inefficiency.
Moreover, none of them consider the straggler problem among devices.
In this paper, we propose \textbf{Layer-wise Staleness} and a novel efficient training algorithm, \textbf{Diversely Stale Parameters} (DSP), which can address all these challenges without loss of accuracy nor memory issue.
We also analyze the convergence of DSP with two popular gradient-based methods and prove that both of them are guaranteed to converge to critical points for non-convex problems.
Finally, extensive experimental results on training deep convolutional neural networks demonstrate that our proposed DSP algorithm can achieve significant training speedup with stronger robustness and better generalization than compared methods.

assistant: text: We propose Diversely Stale Parameters to break lockings of the backpropoagation algorithm and train a CNN in parallel.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Dual-Component Deep Domain Adaptation: A New Approach for Cross Project Software Vulnerability Detection

Paper content:
Owing to the ubiquity of computer software, software vulnerability detection (SVD) has become an important problem in the software industry and in the field of computer security.
One of the most crucial issues in SVD is coping with the scarcity of labeled vulnerabilities in projects that require the laborious manual labeling of code by software security experts.
One possible way to address is to employ deep domain adaptation which has recently witnessed enormous success in transferring learning from structural labeled to unlabeled data sources.
The general idea is to map both source and target data into a joint feature space and close the discrepancy gap of those data in this joint feature space.
Generative adversarial network (GAN) is a technique that attempts to bridge the discrepancy gap and also emerges as a building block to develop deep domain adaptation approaches with state-of-the-art performance.
However, deep domain adaptation approaches using the GAN principle to close the discrepancy gap are subject to the mode collapsing problem that negatively impacts the predictive performance.
Our aim in this paper is to propose Dual Generator-Discriminator Deep Code Domain Adaptation Network (Dual-GD-DDAN) for tackling the problem of transfer learning from labeled to unlabeled software projects in the context of SVD in order to resolve the mode collapsing problem faced in previous approaches.
The experimental results on real-world software projects show that our proposed method outperforms state-of-the-art baselines by a wide margin.

assistant: text: Our aim in this paper is to propose a new approach for tackling the problem of transfer learning from labeled to unlabeled software projects in the context of SVD in order to resolve the mode collapsing problem faced in previous approaches.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Wizard of Wikipedia: Knowledge-Powered Conversational Agents

Paper content:
In open-domain dialogue intelligent agents should exhibit the use of knowledge, however there are few convincing demonstrations of this to date.
The most popular sequence to sequence models typically “generate and hope” generic utterances that can be memorized in the weights of the model when mapping from input utterance(s) to output, rather than employing recalled knowledge as context.
Use of knowledge has so far proved difficult, in part because of the lack of a supervised learning benchmark task which exhibits knowledgeable open dialogue with clear  grounding.
To that end we collect and release a large dataset with conversations directly grounded with knowledge retrieved from Wikipedia.
We then design architectures capable of retrieving knowledge, reading and conditioning on it, and finally generating natural responses.
Our best performing dialogue models are able to conduct knowledgeable discussions on open-domain topics as evaluated by automatic metrics and human evaluations, while our new benchmark allows for measuring further improvements in this important research direction.

assistant: text: We build knowledgeable conversational agents by conditioning on Wikipedia + a new supervised task.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Label Propagation Networks

Paper content:
Graph networks have recently attracted considerable interest, and in particular in the context of semi-supervised learning.
These methods typically work by generating node representations that are propagated throughout a given weighted graph.
Here we argue that for semi-supervised learning, it is more natural to consider propagating labels in the graph instead.
Towards this end, we propose a differentiable neural version of the classic Label Propagation (LP) algorithm.
This formulation can be used for learning edge weights, unlike other methods where weights are set heuristically.
Starting from a layer implementing a single iteration of LP, we proceed by adding several important non-linear steps that significantly enhance the label-propagating mechanism.
Experiments in two distinct settings demonstrate the utility of our approach.

assistant: text: Neural net for graph-based semi-supervised learning; revisits the classics and propagates *labels* rather than feature representations
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: WiSE-ALE: Wide Sample Estimator for Aggregate Latent Embedding

Paper content:
In this paper, we present a new generative model for learning latent embeddings.
Compared to the classical generative process, where each observed data point is generated from an individual latent variable, our approach assumes a global latent variable to generate the whole set of observed data points.
We then propose a learning objective that is derived as an approximation to a lower bound to the data log likelihood, leading to our algorithm, WiSE-ALE.
Compared to the standard ELBO objective, where the variational posterior for each data point is encouraged to match the prior distribution, the WiSE-ALE objective matches the averaged posterior, over all samples, with the prior, allowing the sample-wise posterior distributions to have a wider range of acceptable embedding mean and variance and leading to better reconstruction quality in the auto-encoding process.
Through various examples and comparison to other state-of-the-art VAE models, we demonstrate that WiSE-ALE has excellent information embedding properties, whilst still retaining the ability to learn a smooth, compact representation.

assistant: text: We propose a new latent variable model to learn latent embeddings for some high-dimensional data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Exploiting Invariant Structures for Compression in Neural Networks

Paper content:
Modern neural networks often require deep compositions of high-dimensional nonlinear functions (wide architecture) to achieve high test accuracy, and thus can have overwhelming number of parameters.
Repeated high cost in prediction at test-time makes neural networks ill-suited for devices with constrained memory or computational power.
We introduce an efficient mechanism, reshaped tensor decomposition, to compress neural networks by exploiting three types of invariant structures: periodicity, modulation and low rank.
Our reshaped tensor decomposition method exploits such invariance structures using a technique called tensorization (reshaping the layers into higher-order tensors) combined with higher order tensor decompositions on top of the tensorized layers.
Our compression method improves low rank approximation methods and can be incorporated to (is complementary to) most of the existing compression methods for neural networks to achieve better compression.
Experiments on LeNet-5 (MNIST), ResNet-32 (CI- FAR10) and ResNet-50 (ImageNet) demonstrate that our reshaped tensor decomposition outperforms (5% test accuracy improvement universally on CIFAR10) the state-of-the-art low-rank approximation techniques under same compression rate, besides achieving orders of magnitude faster convergence rates.

assistant: text: Compression of neural networks which improves the state-of-the-art low rank approximation techniques and is complementary to most of other compression techniques.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: AmbientGAN: Generative models from lossy measurements

Paper content:
Generative models provide a way to model structure in complex distributions and have been shown to be useful for many tasks of practical interest.
However, current techniques for training generative models require access to fully-observed samples.
In many settings, it is expensive or even impossible to obtain fully-observed samples, but economical to obtain partial, noisy observations.
We consider the task of learning an implicit generative model given only lossy measurements of samples from the distribution of interest.
We show that the true underlying distribution can be provably recovered even in the presence of per-sample information loss for a class of measurement models.
Based on this, we propose a new method of training Generative Adversarial Networks (GANs) which we call AmbientGAN.
On three benchmark datasets, and for various measurement models, we demonstrate substantial qualitative and quantitative improvements.
Generative models trained with our method can obtain $2$-$4$x higher inception scores than the baselines.

assistant: text: How to learn GANs from noisy, distorted, partial observations
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: At Your Fingertips: Automatic Piano Fingering Detection

Paper content:
Automatic Piano Fingering is a hard task which computers can learn using data.
As data collection is hard and expensive, we propose to automate this process by automatically extracting fingerings from public videos and MIDI files, using computer-vision techniques.
Running this process on 90 videos results in the largest dataset for piano fingering with more than 150K notes.
We show that when running a previously proposed model for automatic piano fingering on our dataset and then fine-tuning it on manually labeled piano fingering data, we achieve state-of-the-art results.
In addition to the fingering extraction method, we also introduce a novel method for transferring deep-learning computer-vision models to work on out-of-domain data, by fine-tuning it on out-of-domain augmentation proposed by a Generative Adversarial Network (GAN).
For demonstration, we anonymously release a visualization of the output of our process for a single video on https://youtu.be/Gfs1UWQhr5Q

assistant: text: We automatically extract fingering information from videos of piano performances, to be used in automatic fingering prediction models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Data-Independent Neural Pruning via Coresets

Paper content:
Previous work showed empirically that large neural networks can be significantly reduced in size while preserving their accuracy.
Model compression became a central research topic, as it is crucial for deployment of neural networks on devices with limited computational and memory resources.
The majority of the compression methods are based on heuristics and offer no worst-case guarantees on the trade-off between the compression rate and the approximation error for an arbitrarily new sample.
We propose the first efficient, data-independent neural pruning algorithm with a provable trade-off between its compression rate and the approximation error for any future test sample.
Our method is based on the coreset framework, which finds a small weighted subset of points that provably approximates the original inputs.
Specifically, we approximate the output of a layer of neurons by a coreset of neurons in the previous layer and discard the rest.
We apply this framework in a layer-by-layer fashion from the top to the bottom.
Unlike previous works, our coreset is data independent, meaning that it provably guarantees the accuracy of the function for any input $x\in \mathbb{R}^d$, including an adversarial one.
We demonstrate the effectiveness of our method on popular network architectures.
In particular, our coresets yield 90% compression of the LeNet-300-100 architecture on MNIST while improving the accuracy.

assistant: text: We propose an efficient, provable and data independent method for network compression via neural pruning using coresets of neurons -- a novel construction proposed in this paper.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Hierarchical Deep Reinforcement Learning Agent with Counter Self-play  on Competitive Games 

Paper content:
Deep Reinforcement Learning algorithms lead to agents that can solve difficult decision making problems in complex environments.
However, many difficult multi-agent competitive games, especially real-time strategy games are still considered beyond the capability of current deep reinforcement learning algorithms, although there has been a recent effort to change this \citep{openai_2017_dota, vinyals_2017_starcraft}.
Moreover, when the opponents in a competitive game are suboptimal, the current \textit{Nash Equilibrium} seeking, self-play algorithms are often unable to generalize their strategies to opponents that play strategies vastly different from their own.
This suggests that a learning algorithm that is beyond conventional self-play is necessary.
We develop Hierarchical Agent with Self-play (HASP), a learning approach for obtaining hierarchically structured policies that can achieve higher performance than conventional self-play on competitive games through the use of a diverse pool of sub-policies we get from Counter Self-Play (CSP).
We demonstrate that the ensemble policy generated by HASP can achieve better performance while facing unseen opponents that use sub-optimal policies.
On a motivating iterated Rock-Paper-Scissor game and a partially observable real-time strategic game (http://generals.io/), we are led to the conclusion that HASP can perform better than conventional self-play as well as achieve 77% win rate against FloBot, an open-source agent which has ranked at position number 2 on the online leaderboards.

assistant: text: We develop Hierarchical Agent with Self-play (HASP), a learning approach for obtaining hierarchically structured policies that can achieve high performance than conventional self-play on competitive real-time strategic games.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Asynchronous Stochastic Subgradient Methods for General Nonsmooth Nonconvex Optimization

Paper content:
Asynchronous distributed methods are a popular way to reduce the communication and synchronization costs of large-scale optimization.
Yet, for all their success, little is known about their convergence guarantees in the challenging case of general non-smooth, non-convex objectives, beyond cases where closed-form proximal operator solutions are available.
This is all the more surprising since these objectives are the ones appearing in the training of deep neural networks.
In this paper, we introduce the first convergence analysis covering asynchronous methods in the case of general non-smooth, non-convex objectives.
Our analysis applies to stochastic sub-gradient descent methods both with and without block variable partitioning, and both with and without momentum.
It is phrased in the context of a general probabilistic model of asynchronous scheduling accurately adapted to modern hardware properties.
We validate our analysis experimentally in the context of training deep neural network architectures.
We show their overall successful asymptotic convergence as well as exploring how momentum, synchronization, and partitioning all affect performance.

assistant: text: Asymptotic convergence for stochastic subgradien method with momentum under general parallel asynchronous computation for general nonconvex nonsmooth optimization