ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: From Adversarial Training to Generative Adversarial Networks

Paper content:
In this paper, we are interested in two seemingly different concepts: \textit{adversarial training} and \textit{generative adversarial networks (GANs)}.
Particularly, how these techniques work to improve each other.
To this end, we analyze the limitation of adversarial training as a defense method, starting from questioning how well the robustness of a model can generalize.
Then, we successfully improve the generalizability via data augmentation by the ``fake'' images sampled from generative adversarial network.
After that, we are surprised to see that the resulting robust classifier leads to a better generator, for free.
We intuitively explain this interesting phenomenon and leave the theoretical analysis for future work.
Motivated by these observations, we propose a system that combines generator, discriminator, and adversarial attacker together in a single network.
After end-to-end training and fine tuning, our method can simultaneously improve the robustness of classifiers, measured by accuracy under strong adversarial attacks, and the quality of generators, evaluated both aesthetically and quantitatively.
In terms of the classifier, we achieve better robustness than the state-of-the-art adversarial training algorithm proposed in (Madry \textit{et al.}, 2017), while our generator achieves competitive performance compared with SN-GAN (Miyato and Koyama, 2018).

assistant: text: We found adversarial training not only speeds up the GAN training but also increases the image quality
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
Progress in understanding how individual animals learn requires high-throughput standardized methods for behavioral training and ways of adapting training.
During the course of training with hundreds or thousands of trials, an animal may change its underlying strategy abruptly, and capturing these changes requires real-time inference of the animal’s latent decision-making strategy.
To address this challenge, we have developed an integrated platform for automated animal training, and an iterative decision-inference model that is able to infer the momentary decision-making policy, and predict the animal’s choice on each trial with an accuracy of ~80\%, even when the animal is performing poorly.
We also combined decision predictions at single-trial resolution with automated pose estimation to assess movement trajectories.
Analysis of these features revealed categories of movement trajectories that associate with decision confidence.
We designed our automated training system as follows (

assistant: text: Automated mice training for neuroscience with online iterative latent strategy inference for behavior prediction
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adaptive Input Representations for Neural Language Modeling

Paper content:
We introduce adaptive input representations for neural language modeling which extend the adaptive softmax of Grave et al. (2017) to input representations of variable capacity.
There are several choices on how to factorize the input and output layers, and whether to model words, characters or sub-word units.
We perform a systematic comparison of popular choices for a self-attentional architecture.
Our experiments show that models equipped with adaptive embeddings are more than twice as fast to train than the popular character input CNN while having a lower number of parameters.
On the WikiText-103 benchmark we achieve 18.7 perplexity, an improvement of 10.5 perplexity compared to the previously best published result and on the Billion Word benchmark, we achieve 23.02 perplexity.

assistant: text: Variable capacity input word embeddings and SOTA on WikiText-103, Billion Word benchmarks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks

Paper content:
This paper is focused on investigating and demystifying an intriguing robustness phenomena in over-parameterized neural network training.
In particular we provide empirical and theoretical evidence that first order methods such as gradient descent are provably robust to noise/corruption on a constant fraction of the labels despite over-parameterization under a rich dataset model.
In particular:
i) First, we show that in the first few iterations where the updates are still in the vicinity of the initialization these algorithms only fit to the correct labels essentially ignoring the noisy labels.
ii) Secondly, we prove that to start to overfit to the noisy labels these algorithms must stray rather far from from the initial model which can only occur after many more iterations.
Together, these show that gradient descent with early stopping is provably robust to label noise and shed light on empirical robustness of deep networks as well as commonly adopted early-stopping heuristics.

assistant: text: We prove that gradient descent is robust to label corruption despite over-parameterization under a rich dataset model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient Training on Very Large Corpora via Gramian Estimation

Paper content:
We study the problem of learning similarity functions over very large corpora using neural network embedding models.
These models are typically trained using SGD with random sampling of unobserved pairs, with a sample size that grows quadratically with the corpus size, making it expensive to scale.
We propose new efficient methods to train these models without having to sample unobserved pairs.
Inspired by matrix factorization, our approach relies on adding a global quadratic penalty and expressing this term as the inner-product of two generalized Gramians.
We show that the gradient of this term can be efficiently computed by maintaining estimates of the Gramians, and develop variance reduction schemes to improve the quality of the estimates.
We conduct large-scale experiments that show a significant improvement both in training time and generalization performance compared to sampling methods.

assistant: text: We develop efficient methods to train neural embedding models with a dot-product structure, by reformulating the objective function in terms of generalized Gram matrices, and maintaining estimates of those matrices.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: An Out-of-the-box Full-network Embedding for Convolutional Neural Networks

Paper content:
Transfer learning for feature extraction can be used to exploit deep representations in contexts where there is very few training data, where there are limited computational resources, or when tuning the hyper-parameters needed for training is not an option.
While previous contributions to feature extraction propose embeddings based on a single layer of the network, in this paper we propose a full-network embedding which successfully integrates convolutional and fully connected features, coming from all layers of a deep convolutional neural network.
To do so, the embedding normalizes features in the context of the problem, and discretizes their values to reduce noise and regularize the embedding space.
Significantly, this also reduces the computational cost of processing the resultant representations.
The proposed method is shown to outperform single layer embeddings on several image classification tasks, while also being more robust to the choice of the pre-trained model used for obtaining the initial features.
The performance gap in classification accuracy between thoroughly tuned solutions and the full-network embedding is also reduced, which makes of the proposed approach a competitive solution for a large set of applications.

assistant: text: We present a full-network embedding of CNN which outperforms single layer embeddings for transfer learning tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: How Powerful are Graph Neural Networks?

Paper content:
Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs.
GNNs follow a neighborhood aggregation scheme, where the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes.
Many GNN variants have been proposed and have achieved state-of-the-art results on both node and graph classification tasks.
However, despite GNNs revolutionizing graph representation learning, there is limited understanding of their representational properties and limitations.
Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures.
Our results characterize the discriminative power of popular GNN variants, such as Graph Convolutional Networks and GraphSAGE, and show that they cannot learn to distinguish certain simple graph structures.
We then develop a simple architecture that is provably the most expressive among the class of GNNs and is as powerful as the Weisfeiler-Lehman graph isomorphism test.
We empirically validate our theoretical findings on a number of graph classification benchmarks, and demonstrate that our model achieves state-of-the-art performance.

assistant: text: We develop theoretical foundations for the expressive power of GNNs and design a provably most powerful GNN.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adversarial Imitation Attack

Paper content:
Deep learning models are known to be vulnerable to adversarial examples.
A practical adversarial attack should require as little as possible knowledge of attacked models T. Current substitute attacks need pre-trained models to generate adversarial examples and their attack success rates heavily rely on the transferability of adversarial examples.
Current score-based and decision-based attacks require lots of queries for the T. In this study, we propose a novel adversarial imitation attack.
First, it produces a replica of the T by a two-player game like the generative adversarial networks (GANs).
The objective of the generative model G is to generate examples which lead D returning different outputs with T. The objective of the discriminative model D is to output the same labels with T under the same inputs.
Then, the adversarial examples generated by D are utilized to fool the T. Compared with the current substitute attacks, imitation attack can use less training data to produce a replica of T and improve the transferability of adversarial examples.
Experiments demonstrate that our imitation attack requires less training data than the black-box substitute attacks, but achieves an attack success rate close to the white-box attack on unseen data with no query.

assistant: text: A novel adversarial imitation attack to fool machine learning models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: The Case for Full-Matrix Adaptive Regularization

Paper content:
Adaptive regularization methods pre-multiply a descent direction by a preconditioning matrix.
Due to the large number of parameters of machine learning problems, full-matrix preconditioning methods are prohibitively expensive.
We show how to modify full-matrix adaptive regularization in order to make it practical and effective.
We also provide novel theoretical analysis
for adaptive regularization in non-convex optimization settings.
The core of our algorithm, termed GGT, consists of efficient inverse computation of square roots of low-rank matrices.
Our preliminary experiments underscore improved convergence rate of GGT across a variety of synthetic tasks and standard deep learning benchmarks.

assistant: text: fast, truly scalable full-matrix AdaGrad/Adam, with theory for adaptive stochastic non-convex optimization
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Energy and Policy Considerations for Deep Learning in NLP

Paper content:
Recent progress in hardware and methodology for training neural networks has ushered in a new generation of large networks trained on abundant data.
These models have obtained notable gains in accuracy across many NLP tasks.
However, these accuracy improvements depend on the availability of exceptionally large computational resources that necessitate similarly substantial energy consumption.
As a result these models are costly to train and develop, both financially, due to the cost of hardware and electricity or cloud compute time, and environmentally, due to the carbon footprint required to fuel modern tensor processing hardware.
In this paper we bring this issue to the attention of NLP researchers by quantifying the approximate financial and environmental costs of training a variety of recently successful neural network models for NLP.
Based on these findings, we propose actionable recommendations to reduce costs and improve equity in NLP research and practice.

assistant: text: We quantify the energy cost in terms of money (cloud credits) and carbon footprint of training recently successful neural network models for NLP. Costs are high.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning-Based Frequency Estimation Algorithms

Paper content:
Estimating the frequencies of elements in a data stream is a fundamental task in data analysis and machine learning.
The problem is typically addressed using streaming algorithms which can process very large data using limited storage.
Today's streaming algorithms, however, cannot exploit patterns in their input to improve performance.
We propose a new class of algorithms that automatically learn relevant patterns in the input data and use them to improve its frequency estimates.
The proposed algorithms combine the benefits of machine learning with the formal guarantees available through algorithm theory.
We prove that our learning-based algorithms have lower estimation errors than their non-learning counterparts.
We also evaluate our algorithms on two real-world datasets and demonstrate empirically their performance gains.

assistant: text: Data stream algorithms can be improved using deep learning, while retaining performance guarantees.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Tasks Without Borders: A New Approach to Online Multi-Task Learning

Paper content:
We introduce MTLAB, a new algorithm for learning multiple related tasks with strong theoretical guarantees.
Its key idea is to perform learning sequentially over the data of all tasks, without interruptions or restarts at task boundaries.
Predictors for individual tasks are derived from this process by an additional online-to-batch conversion step.
By learning across task boundaries, MTLAB achieves a sublinear regret of true risks in the number of tasks.
In the lifelong learning setting, this leads to an improved generalization bound that converges with the total number of samples across all observed tasks, instead of the number of examples per tasks or the number of tasks independently.
At the same time, it is widely applicable: it can handle finite sets of tasks, as common in multi-task learning, as well as stochastic task sequences, as studied in lifelong learning.

assistant: text: A new algorithm for online multi-task learning that learns without restarts at the task borders
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Question Generation from Paragraphs: A Tale of Two Hierarchical Models

Paper content:
Automatic question generation from paragraphs is an important and challenging problem, particularly due to the long context from paragraphs.
In this paper, we propose and study two hierarchical models for the task of question generation from paragraphs.
Specifically, we propose
(a) a novel hierarchical BiLSTM model with selective attention and
(b) a novel hierarchical Transformer architecture, both of which learn hierarchical representations of paragraphs.
We model a paragraph in terms of its constituent sentences, and a sentence in terms of its constituent words.
While the introduction of the attention mechanism benefits the hierarchical BiLSTM model, the hierarchical Transformer, with its inherent attention and positional encoding mechanisms also performs better than flat transformer model.
We conducted empirical evaluation on the widely used SQuAD and MS MARCO datasets using standard metrics.
The results demonstrate the overall effectiveness of the hierarchical models over their flat counterparts.
Qualitatively, our hierarchical models are able to generate fluent and relevant questions.

assistant: text: Automatic question generation from paragraph using hierarchical models
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
We study how the topology of a data set comprising two components representing two classes of objects in a binary classification problem changes as it passes through the layers of a well-trained neural network, i.e., one with perfect accuracy on training set and a generalization error of less than 1%.
The goal is to shed light on two well-known mysteries in deep neural networks: (i) a nonsmooth activation function like ReLU outperforms a smooth one like hyperbolic tangent; (ii) successful neural network architectures rely on having many layers, despite the fact that a shallow network is able to approximate any function arbitrary well.
We performed extensive experiments on persistent homology of a range of point cloud data sets.
The results consistently demonstrate the following: (1) Neural networks operate by changing topology, transforming a topologically complicated data set into a topologically simple one as it passes through the layers.
No matter how complicated the topology of the data set we begin with, when passed through a well-trained neural network, the Betti numbers of both components invariably reduce to their lowest possible values: zeroth Betti number is one and all higher Betti numbers are zero.
Furthermore, (2) the reduction in Betti numbers is significantly faster for ReLU activation compared to hyperbolic tangent activation --- consistent with the fact that the former define nonhomeomorphic maps (that change topology) whereas the latter define homeomorphic maps (that preserve topology).
Lastly, (3) shallow and deep networks process the same data set differently --- a shallow network operates mainly through changing geometry and changes topology only in its final layers, a deep network spreads topological changes more evenly across all its layers.

assistant: text: We show that neural networks operate by changing topologly of a data set and explore how architectural choices effect this change.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: FasterSeg: Searching for Faster Real-time Semantic Segmentation

Paper content:
We present FasterSeg, an automatically designed semantic segmentation network with not only state-of-the-art performance but also faster speed than current methods.
Utilizing neural architecture search (NAS), FasterSeg is discovered from a novel and broader search space integrating multi-resolution branches, that has been recently found to be vital in manually designed segmentation models.
To better calibrate the balance between the goals of high accuracy and low latency, we propose a decoupled and fine-grained latency regularization, that effectively overcomes our observed phenomenons that the searched networks are prone to "collapsing" to low-latency yet poor-accuracy models.
Moreover, we seamlessly extend FasterSeg to a new collaborative search (co-searching) framework, simultaneously searching for a teacher and a student network in the same single run.
The teacher-student distillation further boosts the student model’s accuracy.
Experiments on popular segmentation benchmarks demonstrate the competency of FasterSeg.
For example, FasterSeg can run over 30% faster than the closest manually designed competitor on Cityscapes, while maintaining comparable accuracy.

assistant: text: We present a real-time segmentation model automatically discovered by a multi-scale NAS framework, achieving 30% faster than state-of-the-art models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Natural Language Inference with External Knowledge

Paper content:
Modeling informal inference in natural language is very challenging.
With the recent availability of large annotated data, it has become feasible to train complex models such as neural networks to perform natural language inference (NLI), which have achieved state-of-the-art performance.
Although there exist relatively large annotated data, can machines learn all knowledge needed to perform NLI from the data?
If not, how can NLI models benefit from external knowledge and how to build NLI models to leverage it?
In this paper, we aim to answer these questions by enriching the state-of-the-art neural natural language inference models with external knowledge.
We demonstrate that the proposed models with external knowledge further improve the state of the art on the Stanford Natural Language Inference (SNLI) dataset.

assistant: text: the proposed models with external knowledge further improve the state of the art on the SNLI dataset.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Low Rank Training of Deep Neural Networks for Emerging Memory Technology

Paper content:
The recent success of neural networks for solving difficult decision tasks has incentivized incorporating smart decision making "at the edge."
However, this work has traditionally focused on neural network inference, rather than training, due to memory and compute limitations, especially in emerging non-volatile memory systems, where writes are energetically costly and reduce lifespan.
Yet, the ability to train at the edge is becoming increasingly important as it enables applications such as real-time adaptability to device drift and environmental variation, user customization, and federated learning across devices.
In this work, we address four key challenges for training on edge devices with non-volatile memory: low weight update density, weight quantization, low auxiliary memory, and online learning.
We present a low-rank training scheme that addresses these four challenges while maintaining computational efficiency.
We then demonstrate the technique on a representative convolutional neural network across several adaptation problems, where it out-performs standard SGD both in accuracy and in number of weight updates.

assistant: text: We use Kronecker sum approximations for low-rank training to address challenges in training neural networks on edge devices that utilize emerging memory technologies.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Universal Music Translation Network

Paper content:
We present a method for translating music across musical instruments and styles.
This method is based on unsupervised training of a multi-domain wavenet autoencoder, with a shared encoder and a domain-independent latent space that is trained end-to-end on waveforms.
Employing a diverse training dataset and large net capacity, the single encoder allows us to translate also from musical domains that were not seen during training.
We evaluate our method on a dataset collected from professional musicians, and achieve convincing translations.
We also study the properties of the obtained translation and demonstrate translating even from a whistle, potentially enabling the creation of instrumental music by untrained humans.

assistant: text: An automatic method for converting music between instruments and styles
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: The Frechet Distance of training and test distribution predicts the generalization gap

Paper content:
Learning theory tells us that more data is better when minimizing the generalization error of identically distributed training and test sets.
However, when training and test distribution differ, this distribution shift can have a significant effect.
With a novel perspective on function transfer learning, we are able to lower bound the change of performance when transferring from training to test set with the Wasserstein distance between the embedded training and test set distribution.
We find that there is a trade-off affecting performance between how invariant a function is to changes in training and test distribution and how large this shift in distribution is.
Empirically across several data domains, we substantiate this viewpoint by showing that test performance correlates strongly with the distance in data distributions between training and test set.
Complementary to the popular belief that more data is always better, our results highlight the utility of also choosing a training data distribution that is close to the test data distribution when the learned function is not invariant to such changes.

assistant: text: The Frechet Distance between train and test distribution correlates with the change in performance for functions that are not invariant to the shift.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Unsupervised Few-shot Object Recognition by Integrating Adversarial, Self-supervision, and Deep Metric Learning of Latent Parts

Paper content:
This paper addresses unsupervised few-shot object recognition, where all training images are unlabeled and do not share classes with labeled support images for few-shot recognition in testing.
We use a new GAN-like deep architecture aimed at unsupervised learning of an image representation which will encode latent object parts and thus generalize well to unseen classes in our few-shot recognition task.
Our unsupervised training integrates adversarial, self-supervision, and deep metric learning.
We make two contributions.
First, we extend the vanilla GAN with reconstruction loss to enforce the discriminator capture the most relevant characteristics of "fake" images generated from randomly sampled codes.
Second, we compile a training set of triplet image examples for estimating the triplet loss in metric learning by using an image masking procedure suitably designed to identify latent object parts.
Hence, metric learning ensures that the deep representation of images showing similar object classes which share some parts are closer than the representations of images which do not have common parts.
Our results show that we significantly outperform the state of the art, as well as get similar performance to the common episodic training for fully-supervised few-shot learning on the Mini-Imagenet and Tiered-Imagenet datasets.

assistant: text: We address the problem of unsupervised few-shot object recognition, where all training images are unlabeled and do not share classes with test images.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Character-level Translation with Self-attention

Paper content:
We perform an in-depth investigation of the suitability of self-attention models for character-level neural machine translation.
We test the standard transformer model, as well as a novel variant in which the encoder block combines information from nearby characters using convolution.
We perform extensive experiments on WMT and UN datasets, testing both bilingual and multilingual translation to English using up to three input languages (French, Spanish, and Chinese).
Our transformer variant consistently outperforms the standard transformer at the character-level and converges faster while learning more robust character-level alignments.

assistant: text: We perform an in-depth investigation of the suitability of self-attention models for character-level neural machine translation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep Generative Classifier for Out-of-distribution Sample Detection

Paper content:
The capability of reliably detecting out-of-distribution samples is one of the key factors in deploying a good classifier, as the test distribution always does not match with the training distribution in most real-world applications.
In this work, we propose a deep generative classifier which is effective to detect out-of-distribution samples as well as classify in-distribution samples, by integrating the concept of Gaussian discriminant analysis into deep neural networks.
Unlike the discriminative (or softmax) classifier that only focuses on the decision boundary partitioning its latent space into multiple regions, our generative classifier aims to explicitly model class-conditional distributions as separable Gaussian distributions.
Thereby, we can define the confidence score by the distance between a test sample and the center of each distribution.
Our empirical evaluation on multi-class images and tabular data demonstrate that the generative classifier achieves the best performances in distinguishing out-of-distribution samples, and also it can be generalized well for various types of deep neural networks.

assistant: text: This paper proposes a deep generative classifier which is effective to detect out-of-distribution samples as well as classify in-distribution samples, by integrating the concept of Gaussian discriminant analysis into deep neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Subgradient Descent Learns Orthogonal Dictionaries

Paper content:
This paper concerns dictionary learning, i.e., sparse coding, a fundamental representation learning problem.
We show that a subgradient descent algorithm, with random initialization, can recover orthogonal dictionaries on a natural nonsmooth, nonconvex L1 minimization formulation of the problem, under mild statistical assumption on the data.
This is in contrast to previous provable methods that require either expensive computation or delicate initialization schemes.
Our analysis develops several tools for characterizing landscapes of nonsmooth functions, which might be of independent interest for provable training of deep networks with nonsmooth activations (e.g., ReLU), among other applications.
Preliminary synthetic and real experiments corroborate our analysis and show that our algorithm works well empirically in recovering orthogonal dictionaries.

assistant: text: Efficient dictionary learning by L1 minimization via a novel analysis of the non-convex non-smooth geometry.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Trust-PCL: An Off-Policy Trust Region Method for Continuous Control

Paper content:
Trust region methods, such as TRPO, are often used to stabilize policy optimization algorithms in reinforcement learning (RL).
While current trust region strategies are effective for continuous control, they typically require a large amount of on-policy interaction with the environment.
To address this problem, we propose an off-policy trust region method, Trust-PCL, which exploits an observation that the optimal policy and state values of a maximum reward objective with a relative-entropy regularizer satisfy a set of multi-step pathwise consistencies along any path.
The introduction of relative entropy regularization allows Trust-PCL to maintain optimization stability while exploiting off-policy data to improve sample efficiency.
When evaluated on a number of continuous control tasks, Trust-PCL significantly improves the solution quality and sample efficiency of TRPO.

assistant: text: We extend recent insights related to softmax consistency to achieve state-of-the-art results in continuous control.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: FlowQA: Grasping Flow in History for Conversational Machine Comprehension

Paper content:
Conversational machine comprehension requires a deep understanding of the conversation history.
To enable traditional, single-turn models to encode the history comprehensively, we introduce Flow, a mechanism that can incorporate intermediate representations generated during the process of answering previous questions, through an alternating parallel processing structure.
Compared to shallow approaches that concatenate previous questions/answers as input, Flow integrates the latent semantics of the conversation history more deeply.
Our model, FlowQA, shows superior performance on two recently proposed conversational challenges (+7.2% F1 on CoQA and +4.0% on QuAC).
The effectiveness of Flow also shows in other tasks.
By reducing sequential instruction understanding to conversational machine comprehension, FlowQA outperforms the best models on all three domains in SCONE, with +1.8% to +4.4% improvement in accuracy.

assistant: text: We propose the Flow mechanism and an end-to-end architecture, FlowQA, that achieves SotA on two conversational QA datasets and a sequential instruction understanding task.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Bayesian Model Selection for Identifying Markov Equivalent Causal Graphs

Paper content:
Many approaches to causal discovery are limited by their inability to discriminate between Markov equivalent graphs given only observational data.
We formulate causal discovery as a marginal likelihood based Bayesian model selection problem.
We adopt a parameterization based on the notion of the independence of causal mechanisms which renders Markov equivalent graphs distinguishable.
We complement this with an empirical Bayesian approach to setting priors so that the actual underlying causal graph is assigned a higher marginal likelihood than its alternatives.
Adopting a Bayesian approach also allows for straightforward modeling of unobserved confounding variables, for which we provide a variational algorithm to approximate the marginal likelihood, since this desirable feat renders the computation of the marginal likelihood intractable.
We believe that the Bayesian approach to causal discovery both allows the rich methodology of Bayesian inference to be used in various difficult aspects of this problem and provides a unifying framework to causal discovery research.
We demonstrate promising results in experiments conducted on real data, supporting our modeling approach and our inference methodology.

assistant: text: We cast causal structure discovery as a Bayesian model selection in a way that allows us to discriminate between Markov equivalent graphs to identify the unique causal graph.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: FACE SUPER-RESOLUTION GUIDED BY 3D FACIAL PRIORS

Paper content:
State-of-the-art face super-resolution methods employ deep convolutional neural networks to learn a mapping between low- and high-resolution facial patterns by exploring local appearance knowledge.
However, most of these methods do not well exploit facial structures and identity information, and struggle to deal with facial images that exhibit large pose variation and misalignment.
In this paper, we propose a novel face super-resolution method that explicitly incorporates 3D facial priors which grasp the sharp facial structures.
Firstly, the 3D face rendering branch is set up to obtain 3D priors of salient facial structures and identity knowledge.
Secondly, the Spatial Attention Mechanism is used to better exploit this hierarchical information (i.e. intensity similarity, 3D facial structure, identity content) for the super-resolution problem.
Extensive experiments demonstrate that the proposed algorithm achieves superior face super-resolution results and outperforms the state-of-the-art.

assistant: text: We propose a novel face super resolution method that explicitly incorporates 3D facial priors which grasp the sharp facial structures.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Neural Stored-program Memory

Paper content:
Neural networks powered with external memory simulate computer behaviors.
These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks.
In this paper, we introduce a new memory to store weights for the controller, analogous to the stored-program memory in modern computer architectures.
The proposed model, dubbed Neural Stored-program Memory, augments current memory-augmented neural networks, creating differentiable machines that can switch programs through time, adapt to variable contexts and thus fully resemble the Universal Turing Machine.
A wide range of experiments demonstrate that the resulting machines not only excel in classical algorithmic problems, but also have potential for compositional, continual, few-shot learning and question-answering tasks.

assistant: text: A neural simulation of Universal Turing Machine
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: BERT for Sequence-to-Sequence Multi-Label Text Classification

Paper content:
We study the BERT language representation model and the sequence generation model with BERT encoder for multi-label text classification task.
We experiment with both models and explore their special qualities for this setting.
We also introduce and examine experimentally a mixed model, which is an ensemble of multi-label BERT and sequence generating BERT models.
Our experiments demonstrated that BERT-based models and the mixed model, in particular, outperform current baselines in several metrics achieving state-of-the-art results on three well-studied multi-label classification datasets with English texts and two private Yandex Taxi datasets with Russian texts.

assistant: text: On using BERT as an encoder for sequential prediction of labels in multi-label text classification task
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks

Paper content:
Deep learning models are vulnerable to adversarial examples crafted by applying human-imperceptible perturbations on benign inputs.
However, under the black-box setting, most existing adversaries often have a poor transferability to attack other defense models.
In this work, from the perspective of regarding the adversarial example generation as an optimization process, we propose two new methods to improve the transferability of adversarial examples, namely Nesterov Iterative Fast Gradient Sign Method (NI-FGSM) and Scale-Invariant attack Method (SIM).
NI-FGSM aims to adapt Nesterov accelerated gradient into the iterative attacks so as to effectively look ahead and improve the transferability of adversarial examples.
While SIM is based on our discovery on the scale-invariant property of deep learning models, for which we leverage to optimize the adversarial perturbations over the scale copies of the input images so as to avoid "overfitting” on the white-box model being attacked and generate more transferable adversarial examples.
NI-FGSM and SIM can be naturally integrated to build a robust gradient-based attack to generate more transferable adversarial examples against the defense models.
Empirical results on ImageNet dataset demonstrate that our attack methods exhibit higher transferability and achieve higher attack success rates than state-of-the-art gradient-based attacks.

assistant: text: We proposed a Nesterov Iterative Fast Gradient Sign Method (NI-FGSM) and a Scale-Invariant attack Method (SIM) that can boost the transferability of adversarial examples for image classification.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Optimizing the Latent Space of Generative Networks

Paper content:
Generative Adversarial Networks (GANs) have achieved remarkable results in the task of generating realistic natural images.
In most applications, GAN models share two aspects in common.
On the one hand, GANs training involves solving a challenging saddle point optimization problem, interpreted as an adversarial game between a generator and a discriminator functions.
On the other hand, the generator and the discriminator are parametrized in terms of deep convolutional neural networks.
The goal of this paper is to disentangle the contribution of these two factors to the success of GANs.
In particular, we introduce Generative Latent Optimization (GLO), a framework to train deep convolutional generators without using discriminators, thus avoiding the instability of adversarial optimization problems.
Throughout a variety of experiments, we show that GLO enjoys many of the desirable properties of GANs: learning from large data, synthesizing visually-appealing samples, interpolating meaningfully between samples, and performing linear arithmetic with noise vectors.

assistant: text: Are GANs successful because of adversarial training or the use of ConvNets? We show a ConvNet generator trained with a simple reconstruction loss and learnable noise vectors leads many of the desirable properties of a  GAN.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: GMM-UNIT: Unsupervised Multi-Domain and Multi-Modal Image-to-Image Translation via Attribute Gaussian Mixture Modelling

Paper content:
Unsupervised image-to-image translation aims to learn a mapping between several visual domains by using unpaired training pairs.
Recent studies have shown remarkable success in image-to-image translation for multiple domains but they suffer from two main limitations: they are either built from several two-domain mappings that are required to be learned independently and/or they generate low-diversity results, a phenomenon known as model collapse.
To overcome these limitations, we propose a method named GMM-UNIT based on a content-attribute disentangled representation, where the attribute space is fitted with a GMM.
Each GMM component represents a domain, and this simple assumption has two prominent advantages.
First, the dimension of the attribute space does not grow linearly with the number of domains, as it is the case in the literature.
Second, the continuous domain encoding allows for interpolation between domains and for extrapolation to unseen domains.
Additionally, we show how GMM-UNIT can be constrained down to different methods in the literature, meaning that GMM-UNIT is a unifying framework for unsupervised image-to-image translation.

assistant: text: GMM-UNIT is an image-to-image translation model that maps an image to multiple domains in a stochastic fashion.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Unsupervised Deep Structure Learning by Recursive Dependency Analysis

Paper content:
We introduce an unsupervised structure learning algorithm for deep, feed-forward, neural networks.
We propose a new interpretation for depth and inter-layer connectivity where a hierarchy of independencies in the input distribution is encoded in the network structure.
This results in structures allowing neurons to connect to neurons in any deeper layer skipping intermediate layers.
Moreover, neurons in deeper layers encode low-order (small condition sets) independencies and have a wide scope of the input, whereas neurons in the first layers encode higher-order (larger condition sets) independencies and have a narrower scope.
Thus, the depth of the network is automatically determined---equal to the maximal order of independence in the input distribution, which is the recursion-depth of the algorithm.
The proposed algorithm constructs two main graphical models:
1) a generative latent graph (a deep belief network) learned from data and
2) a deep discriminative graph constructed from the generative latent graph.
We prove that conditional dependencies between the nodes in the learned generative latent graph are preserved in the class-conditional discriminative graph.
Finally, a deep neural network structure is constructed based on the discriminative graph.
We demonstrate on image classification benchmarks that the algorithm replaces the deepest layers (convolutional and dense layers) of common convolutional networks, achieving high classification accuracy, while constructing significantly smaller structures.
The proposed structure learning algorithm requires a small computational cost and runs efficiently on a standard desktop CPU.

assistant: text: A principled approach for structure learning of deep neural networks with a new interpretation for depth and inter-layer connectivity.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Explaining Time Series by Counterfactuals

Paper content:
We propose a method to automatically compute the importance of features at every observation in time series, by simulating counterfactual trajectories given previous observations.
We define the importance of each observation as the change in the model output caused by replacing the observation with a generated one.
Our method can be applied to arbitrarily complex time series models.
We compare the generated feature importance to existing methods like sensitivity analyses, feature occlusion, and other explanation baselines to show that our approach generates more precise explanations and is less sensitive to noise in the input signals.

assistant: text: Explaining Multivariate Time Series Models by finding important observations in time using Counterfactuals
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep 3D Pan via Local adaptive "t-shaped" convolutions with global and local adaptive dilations

Paper content:
Recent advances in deep learning have shown promising results in many low-level vision tasks.
However, solving the single-image-based view synthesis is still an open problem.
In particular, the generation of new images at parallel camera views given a single input image is of great interest, as it enables 3D visualization of the 2D input scenery.
We propose a novel network architecture to perform stereoscopic view synthesis at arbitrary camera positions along the X-axis, or Deep 3D Pan, with "t-shaped" adaptive kernels equipped with globally and locally adaptive dilations.
Our proposed network architecture, the monster-net, is devised with a novel t-shaped adaptive kernel with globally and locally adaptive dilation, which can efficiently incorporate global camera shift into and handle local 3D geometries of the target image's pixels for the synthesis of naturally looking 3D panned views when a 2-D input image is given.
Extensive experiments were performed on the KITTI, CityScapes and our VXXLXX_STEREO indoors dataset to prove the efficacy of our method.
Our monster-net significantly outperforms the state-of-the-art method, SOTA, by a large margin in all metrics of RMSE, PSNR, and SSIM.
Our proposed monster-net is capable of reconstructing more reliable image structures in synthesized images with coherent geometry.
Moreover, the disparity information that can be extracted from the "t-shaped" kernel is much more reliable than that of the SOTA for the unsupervised monocular depth estimation task, confirming the effectiveness of our method.

assistant: text: Novel architecture for stereoscopic view synthesis at arbitrary camera shifts utilizing adaptive t-shaped kernels with adaptive dilations.