ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: CrescendoNet: A Simple Deep Convolutional Neural Network with Ensemble Behavior

Paper content:
We introduce a new deep convolutional neural network, CrescendoNet, by stacking simple building blocks without residual connections.
Each Crescendo block contains independent convolution paths with increased depths.
The numbers of convolution layers and parameters are only increased linearly in Crescendo blocks.
In experiments, CrescendoNet with only 15 layers outperforms almost all networks without residual connections on benchmark datasets, CIFAR10, CIFAR100, and SVHN.
Given sufficient amount of data as in SVHN dataset, CrescendoNet with 15 layers and 4.1M parameters can match the performance of DenseNet-BC with 250 layers and 15.3M parameters.
CrescendoNet provides a new way to construct high performance deep convolutional neural networks without residual connections.
Moreover, through investigating the behavior and performance of subnetworks in CrescendoNet, we note that the high performance of CrescendoNet may come from its implicit ensemble behavior, which differs from the FractalNet that is also a deep convolutional neural network without residual connections.
Furthermore, the independence between paths in CrescendoNet allows us to introduce a new path-wise training procedure, which can reduce the memory needed for training.

assistant: text: We introduce CrescendoNet, a deep CNN architecture by stacking simple building blocks without residual connections.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient Bayesian Inference for Nested Simulators

Paper content:
We introduce two approaches for conducting efficient Bayesian inference in stochastic simulators containing nested stochastic sub-procedures, i.e., internal procedures for which the density cannot be calculated directly such as rejection sampling loops.
The resulting class of simulators are used extensively throughout the sciences and can be interpreted as probabilistic generative models.
However, drawing inferences from them poses a substantial challenge due to the inability to evaluate even their unnormalised density, preventing the use of many standard inference procedures like Markov Chain Monte Carlo (MCMC).
To address this, we introduce inference algorithms based on a two-step approach that first approximates the conditional densities of the individual sub-procedures, before using these approximations to run MCMC methods on the full program.
Because the sub-procedures can be dealt with separately and are lower-dimensional than that of the overall problem, this two-step process allows them to be isolated and thus be tractably dealt with, without placing restrictions on the overall dimensionality of the problem.
We demonstrate the utility of our approach on a simple, artificially constructed simulator.

assistant: text: We introduce two approaches for efficient and scalable inference in stochastic simulators for which the density cannot be evaluated directly due to, for example, rejection sampling loops.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: DeepEnFM: Deep neural networks with Encoder enhanced Factorization Machine

Paper content:
Click Through Rate (CTR) prediction is a critical task in industrial applications, especially for online social and commerce applications.
It is challenging to find a proper way to automatically discover the effective cross features in CTR tasks.
We propose a novel model for CTR tasks, called Deep neural networks with Encoder enhanced Factorization Machine (DeepEnFM).
Instead of learning the cross features directly, DeepEnFM adopts the Transformer encoder as a backbone to align the feature embeddings with the clues of other fields.
The embeddings generated from encoder are beneficial for the further feature interactions.
Particularly, DeepEnFM utilizes a bilinear approach to generate different similarity functions with respect to different field pairs.
Furthermore, the max-pooling method makes DeepEnFM feasible to capture both the supplementary and suppressing information among different attention heads.
Our model is validated on the Criteo and Avazu datasets, and achieves state-of-art performance.

assistant: text: DNN and Encoder enhanced FM with bilinear attention and max-pooling for CTR
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: PAIRWISE AUGMENTED GANS WITH ADVERSARIAL RECONSTRUCTION LOSS

Paper content:
We propose a novel autoencoding model called Pairwise Augmented GANs.
We train a generator and an encoder jointly and in an adversarial manner.
The generator network learns to sample realistic objects.
In turn, the encoder network at the same time is trained to map the true data distribution to the prior in  latent space.
To ensure good reconstructions, we introduce an augmented adversarial reconstruction loss.
Here we train a discriminator to distinguish two types of pairs: an object with its augmentation and the one with its reconstruction.
We show that such adversarial loss compares objects based on the content rather than on the exact match.
We experimentally demonstrate that our model generates samples and reconstructions of quality competitive with state-of-the-art on datasets MNIST, CIFAR10, CelebA and achieves good quantitative results on CIFAR10.

assistant: text: We propose a novel autoencoding model with augmented adversarial reconstruction loss. We intoduce new metric for content-based assessment of reconstructions.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Robust Goal Recognition with Operator-Counting Heuristics

Paper content:
Goal recognition is the problem of inferring the correct goal towards which an agent executes a plan, given a set of goal hypotheses, a domain model, and a (possibly noisy) sample of the plan being executed.
This is a key problem in both cooperative and competitive agent interactions and recent approaches have produced fast and accurate goal recognition algorithms.
In this paper, we leverage advances in operator-counting heuristics computed using linear programs over constraints derived from classical planning problems to solve goal recognition problems.
Our approach uses additional operator-counting constraints derived from the observations to efficiently infer the correct goal, and serves as basis for a number of further methods with additional constraints.

assistant: text: A goal recognition approach based on operator counting heuristics used to account for noise in the dataset.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Design for Interpretability

Paper content:
The interpretability of an AI agent's behavior is of utmost importance for effective human-AI interaction.
To this end, there has been increasing interest in characterizing and generating interpretable behavior of the agent.
An alternative approach to guarantee that the agent generates interpretable behavior would be to design the agent's environment such that uninterpretable behaviors are either prohibitively expensive or unavailable to the agent.
To date, there has been work under the umbrella of goal or plan recognition design exploring this notion of environment redesign in some specific instances of interpretable of behavior.
In this position paper, we scope the landscape of interpretable behavior and environment redesign in all its different flavors.
Specifically, we focus on three specific types of interpretable behaviors -- explicability, legibility, and predictability -- and present a general framework for the problem of environment design that can be instantiated to achieve each of the three interpretable behaviors.
We also discuss how specific instantiations of this framework correspond to prior works on environment design and identify exciting opportunities for future work.

assistant: text: We present an approach to redesign the environment such that uninterpretable agent behaviors are minimized or eliminated.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: The Blessing of Dimensionality: An Empirical Study of Generalization

Paper content:
The power of neural networks lies in their ability to generalize to unseen data, yet the underlying reasons for this phenomenon remain elusive.
Numerous rigorous attempts have been made to explain generalization, but available bounds are still quite loose, and analysis does not always lead to true understanding.
The goal of this work is to make generalization more intuitive.
Using visualization methods, we discuss the mystery of generalization, the geometry of loss landscapes, and how the curse (or, rather, the blessing) of dimensionality causes optimizers to settle into minima that generalize well.

assistant: text: An intuitive empirical and visual exploration of the generalization properties of deep neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Evaluation of generative networks through their data augmentation capacity

Paper content:
Generative networks are known to be difficult to assess.
Recent works on generative models, especially on generative adversarial networks, produce nice samples of varied categories of images.
But the validation of their quality is highly dependent on the method used.
A good generator should generate data which contain meaningful and varied information and that fit the distribution of a dataset.
This paper presents a new method to assess a generator.
Our approach is based on training a classifier with a mixture of real and generated samples.
We train a generative model over a labeled training set, then we use this generative model to sample new data points that we mix with the original training data.
This mixture of real and generated data is thus used to train a classifier which is afterwards tested on a given labeled test dataset.
We compare this result with the score of the same classifier trained on the real training data mixed with noise.
By computing the classifier's accuracy with different ratios of samples from both distributions (real and generated) we are able to estimate if the generator successfully fits and is able to generalize the distribution of the dataset.
Our experiments compare the result of different generators from the VAE and GAN framework on MNIST and fashion MNIST dataset.

assistant: text: Evaluating generative networks through their data augmentation capacity on discrimative models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning To Generate Reviews and Discovering Sentiment

Paper content:
We explore the properties of byte-level recurrent language models.
When given sufficient amounts of capacity, training data, and compute time, the representations learned by these models include disentangled features corresponding to high-level concepts.
Specifically, we find a single unit which performs sentiment analysis.
These representations, learned in an unsupervised manner, achieve state of the art on the binary subset of the Stanford Sentiment Treebank.
They are also very data efficient.
When using only a handful of labeled examples, our approach matches the performance of strong baselines trained on full datasets.
We also demonstrate the sentiment unit has a direct influence on the generative process of the model.
Simply fixing its value to be positive or negative generates samples with the corresponding positive or negative sentiment.

assistant: text: Byte-level recurrent language models learn high-quality domain specific representations of text.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Imagining the Latent Space of a Variational Auto-Encoders

Paper content:
Variational Auto-Encoders (VAEs) are designed to capture compressible information about a dataset.
As a consequence the information stored in the latent space is seldom sufficient to reconstruct a particular image.
To help understand the type of information stored in the latent space we train a GAN-style decoder constrained to produce images that the VAE encoder will map to the same region of latent space.
This allows us to ''imagine'' the information captured in the latent space.
We argue that this is necessary to make a VAE into a truly generative model.
We use our GAN to visualise the latent space of a standard VAE and of a $\beta$-VAE.

assistant: text: To understand the information stored in the latent space, we train a GAN-style decoder constrained to produce images that the VAE encoder will map to the same region of latent space.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Monge-Amp\`ere Flow for Generative Modeling

Paper content:
We present a deep generative model, named Monge-Amp\`ere flow, which builds on continuous-time gradient flow arising from the Monge-Amp\`ere equation in optimal transport theory.
The generative map from the latent space to the data space follows a dynamical system, where a learnable potential function guides a compressible fluid to flow towards the target density distribution.
Training of the model amounts to solving an optimal control problem.
The Monge-Amp\`ere flow has tractable likelihoods and supports efficient sampling and inference.
One can easily impose symmetry constraints in the generative model by designing suitable scalar potential functions.
We apply the approach to unsupervised density estimation of the MNIST dataset and variational calculation of the two-dimensional Ising model at the critical point.
This approach brings insights and techniques from Monge-Amp\`ere equation, optimal transport, and fluid dynamics into reversible flow-based generative models.

assistant: text: A gradient flow based dynamical system for invertible generative modeling
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Counterfactuals uncover the modular structure of deep generative models

Paper content:
Deep generative models can emulate the perceptual properties of complex image datasets, providing a latent representation of the data.
However, manipulating such representation to perform meaningful and controllable transformations in the data space remains challenging without some form of supervision.
While previous work has focused on exploiting statistical independence to \textit{disentangle} latent factors, we argue that such requirement can be advantageously relaxed and propose instead a non-statistical framework that relies on identifying a modular organization of the network, based on counterfactual manipulations.
Our experiments support that modularity between groups of channels is achieved to a certain degree on a variety of generative models.
This allowed the design of targeted interventions on complex image datasets, opening the way to applications such as computationally efficient style transfer and the automated assessment of robustness to contextual changes in pattern recognition systems.

assistant: text: We develop a framework to find modular internal representations in generative models and manipulate then to generate counterfactual examples.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic

Paper content:
Learning a policy using only observational data is challenging because the distribution of states it induces at execution time may differ from the distribution observed during training.
In this work, we propose to train a policy while explicitly penalizing the mismatch between these two distributions over a fixed time horizon.
We do this by using a learned model of the environment dynamics which is unrolled for multiple time steps, and training a policy network to minimize a differentiable cost over this rolled-out trajectory.
This cost contains two terms: a policy cost which represents the objective the policy seeks to optimize, and an uncertainty cost which represents its divergence from the states it is trained on.
We propose to measure this second cost by using the uncertainty of the dynamics model about its own predictions, using recent ideas from uncertainty estimation for deep networks.
We evaluate our approach using a large-scale observational dataset of driving behavior recorded from traffic cameras, and show that we are able to learn effective driving policies from purely observational data, with no environment interaction.

assistant: text: A model-based RL approach which uses a differentiable uncertainty penalty to learn driving policies from purely observational data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On the Convergence of Adam and Beyond

Paper content:
Several recently proposed stochastic optimization methods that have been successfully used in training deep networks such as RMSProp, Adam, Adadelta, Nadam are based on using gradient updates scaled by square roots of exponential moving averages of squared past gradients.
In many applications, e.g. learning with large output spaces, it has been empirically observed that these algorithms fail to converge to an optimal solution (or a critical point in nonconvex settings).
We show that one cause for such failures is the exponential moving average used in the algorithms.
We provide an explicit example of a simple convex optimization setting where Adam does not converge to the optimal solution, and describe the precise problems with the previous analysis of Adam algorithm.
Our analysis suggests that the convergence issues can be fixed by endowing such algorithms with ``long-term memory'' of past gradients, and propose new variants of the Adam algorithm which not only fix the convergence issues but often also lead to improved empirical performance.

assistant: text: We investigate the convergence of popular optimization algorithms like Adam , RMSProp and propose new variants of these methods which provably converge to optimal solution in convex  settings.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Equi-normalization of Neural Networks

Paper content:
Modern neural networks are over-parametrized.
In particular, each rectified linear hidden unit can be modified by a multiplicative factor by adjusting input and out- put weights, without changing the rest of the network.
Inspired by the Sinkhorn-Knopp algorithm, we introduce a fast iterative method for minimizing the l2 norm of the weights, equivalently the weight decay regularizer.
It provably converges to a unique solution.
Interleaving our algorithm with SGD during training improves the test accuracy.
For small batches, our approach offers an alternative to batch- and group- normalization on CIFAR-10 and ImageNet with a ResNet-18.

assistant: text: Fast iterative algorithm to balance the energy of a network while staying in the same functional equivalence class
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Deep Generative Models of Graphs

Paper content:
Graphs are fundamental data structures required to model many important real-world data, from knowledge graphs, physical and social interactions to molecules and proteins.
In this paper, we study the problem of learning generative models of graphs from a dataset of graphs of interest.
After learning, these models can be used to generate samples with similar properties as the ones in the dataset.
Such models can be useful in a lot of applications, e.g. drug discovery and knowledge graph construction.
The task of learning generative models of graphs, however, has its unique challenges.
In particular, how to handle symmetries in graphs and ordering of its elements during the generation process are important issues.
We propose a generic graph neural net based model that is capable of generating any arbitrary graph.
We study its performance on a few graph generation tasks compared to baselines that exploit domain knowledge.
We discuss potential issues and open problems for such generative models going forward.

assistant: text: We study the graph generation problem and propose a powerful deep generative model capable of generating arbitrary graphs.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Distribution Matching Prototypical Network for Unsupervised Domain Adaptation

Paper content:
State-of-the-art Unsupervised Domain Adaptation (UDA) methods learn transferable features by minimizing the feature distribution discrepancy between the source and target domains.
Different from these methods which do not model the feature distributions explicitly, in this paper, we explore explicit feature distribution modeling for UDA.
In particular, we propose Distribution Matching Prototypical Network (DMPN) to model the deep features from each domain as Gaussian mixture distributions.
With explicit feature distribution modeling, we can easily measure the discrepancy between the two domains.
In DMPN, we propose two new domain discrepancy losses with probabilistic interpretations.
The first one minimizes the distances between the corresponding Gaussian component means of the source and target data.
The second one minimizes the pseudo negative log likelihood of generating the target features from source feature distribution.
To learn both discriminative and domain invariant features, DMPN is trained by minimizing the classification loss on the labeled source data and the domain discrepancy losses together.
Extensive experiments are conducted over two UDA tasks.
Our approach yields a large margin in the Digits Image transfer task over state-of-the-art approaches.
More remarkably, DMPN obtains a mean accuracy of 81.4% on VisDA 2017 dataset.
The hyper-parameter sensitivity analysis shows that our approach is robust w.r.t hyper-parameter changes.

assistant: text: We propose to explicitly model deep feature distributions of source and target data as Gaussian mixture distributions for Unsupervised Domain Adaptation (UDA) and achieve superior results in multiple UDA tasks than state-of-the-art methods.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On recognition of Cyrillic Text

Paper content:
We introduce the largest (among publicly available) dataset for Cyrillic Handwritten Text Recognition and the first dataset for Cyrillic Text in the Wild Recognition, as well as suggest a method for recognizing Cyrillic Handwritten Text and Text in the Wild.
Based on this approach, we develop a system that can reduce the document processing time for one of the largest mathematical competitions in Ukraine by 12 days and the amount of used paper by 0.5 ton.

assistant: text: We introduce several datasets for Cyrillic OCR and a method for its recognition
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: No Spurious Local Minima in a Two Hidden Unit ReLU Network

Paper content:
Deep learning models can be efficiently optimized via stochastic gradient descent, but there is little theoretical evidence to support this.
A key question in optimization is to understand when the optimization landscape of a neural network is amenable to gradient-based optimization.
We focus on a simple neural network two-layer ReLU network with two hidden units, and show that all local minimizers are global.
This combined with recent work of Lee et al. (2017); Lee et al. (2016) show that  gradient descent converges to the global minimizer.

assistant: text: Recovery guarantee of stochastic gradient descent with random initialization for learning a two-layer neural network with two hidden nodes, unit-norm weights, ReLU activation functions and Gaussian inputs.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generalized Transformation-based Gradient

Paper content:
The reparameterization trick has become one of the most useful tools in the field of variational inference.
However, the reparameterization trick is based on the standardization transformation which restricts the scope of application of this method to distributions that have tractable inverse cumulative distribution functions or are expressible as deterministic transformations of such distributions.
In this paper, we generalized the reparameterization trick by allowing a general transformation.
Unlike other similar works, we develop the generalized transformation-based gradient model formally and rigorously.
We discover that the proposed model is a special case of control variate indicating that the proposed model can combine the advantages of CV and generalized reparameterization.
Based on the proposed gradient model, we propose a new polynomial-based gradient estimator which has better theoretical performance than the reparameterization trick under certain condition and can be applied to a larger class of variational distributions.
In studies of synthetic and real data, we show that our proposed gradient estimator has a significantly lower gradient variance than other state-of-the-art methods thus enabling a faster inference procedure.

assistant: text: We propose a novel generalized transformation-based gradient model and propose a polynomial-based gradient estimator based upon the model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Meta-Learning for Low-Resource Neural Machine Translation

Paper content:
In this paper, we propose to extend the recently introduced model-agnostic meta-learning algorithm (MAML, Finn et al., 2017) for low resource neural machine translation (NMT).
We frame low-resource translation as a meta-learning problem, and we learn to adapt to low-resource languages based on multilingual high-resource language tasks.
We use the universal lexical representation (Gu et al., 2018b) to overcome the input-output mismatch across different languages.
We evaluate the proposed meta-learning strategy using eighteen European languages (Bg, Cs, Da, De, El, Es, Et, Fr, Hu, It, Lt, Nl, Pl, Pt, Sk, Sl, Sv and Ru) as source tasks and five diverse languages (Ro, Lv, Fi, Tr, and Ko) as target tasks.
We show that the proposed approach significantly outperforms the multilingual, transfer learning based approach (Zoph et al., 2016) and enables us to train a competitive NMT system with only a fraction of training examples.
For instance, the proposed approach can achieve as high as 22.04 BLEU on Romanian-English WMT’16 by seeing only 16,000 translated words (~600 parallel sentences).

assistant: text: we propose a meta-learning approach for low-resource neural machine translation that can rapidly learn to translate on a new language
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Pattern Selection for Optimal Classical Planning with Saturated Cost Partitioning

Paper content:
Pattern databases are the foundation of some of the strongest admissible heuristics for optimal classical planning.
Experiments showed that the most informative way of combining information from multiple pattern databases is to use saturated cost partitioning.
Previous work selected patterns and computed saturated cost partitionings over the resulting pattern database heuristics in two separate steps.
We introduce a new method that uses saturated cost partitioning to select patterns and show that it outperforms all existing pattern selection algorithms.

assistant: text: Using saturated cost partitioning to select patterns is preferable to all existing pattern selection algorithms.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Shape Features Improve General Model Robustness

Paper content:
Recent studies show that convolutional neural networks (CNNs) are vulnerable under various settings, including adversarial examples, backdoor attacks, and distribution shifting.
Motivated by the findings that human visual system pays more attention to global structure (e.g., shape) for recognition while CNNs are biased towards local texture features in images, we propose a unified framework EdgeGANRob based on robust edge features to improve the robustness of CNNs in general, which first explicitly extracts shape/structure features from a given image and then reconstructs a new image by refilling the texture information with a trained generative adversarial network (GAN).
In addition, to reduce the sensitivity of edge detection algorithm to adversarial perturbation, we propose a robust edge detection approach Robust Canny based on the vanilla Canny algorithm.
To gain more insights, we also compare EdgeGANRob with its simplified backbone procedure EdgeNetRob, which performs learning tasks directly on the extracted robust edge features.
We find that EdgeNetRob can help boost model robustness significantly but at the cost of the clean model accuracy.
EdgeGANRob, on the other hand, is able to improve clean model accuracy compared with EdgeNetRob and without losing the robustness benefits introduced by EdgeNetRob.
Extensive experiments show that EdgeGANRob is resilient in different learning tasks under diverse settings.

assistant: text: A unified  model  to improve model robustness against multiple tasks
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Neural Network Compression using Transform Coding and Clustering

Paper content:
With the deployment of neural networks on mobile devices and the necessity of transmitting neural networks over limited or expensive channels, the file size of trained model was identified as bottleneck.
We propose a codec for the compression
of neural networks which is based on transform coding for convolutional and dense layers and on clustering for biases and normalizations.
With this codec, we achieve average compression factors between 7.9–9.3 while the accuracy of the compressed networks for image classification decreases only by 1%–2%, respectively.

assistant: text: Our neural network codec (which is based on transform coding and clustering) enables a low complexity and high efficient transparent compression of neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Sparse Latent Representations with the Deep Copula Information Bottleneck

Paper content:
Deep latent variable models are powerful tools for representation learning.
In this paper, we adopt the deep information bottleneck model, identify its shortcomings and propose a model that circumvents them.
To this end, we apply a copula transformation which, by restoring the invariance properties of the information bottleneck method, leads to disentanglement of the features in the latent space.
Building on that, we show how this transformation translates to sparsity of the latent space in the new model.
We evaluate our method on artificial and real data.

assistant: text: We apply the copula transformation to the Deep Information Bottleneck which leads to restored invariance properties and a disentangled latent space with superior predictive capabilities.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adaptive Memory Networks

Paper content:
Real-world Question Answering (QA) tasks consist of thousands of words that often represent many facts and entities.
Existing models based on LSTMs require a large number of parameters to support external memory and do not generalize well for long sequence inputs.
Memory networks attempt to address these limitations by storing information to an external memory module but must examine all inputs in the memory.
Hence, for longer sequence inputs the intermediate memory components proportionally scale in size resulting in poor inference times and high computation costs.
In this paper, we present Adaptive Memory Networks (AMN) that process input question pairs to dynamically construct a network architecture optimized for lower inference times.
During inference, AMN parses input text into entities within different memory slots.
However, distinct from previous approaches, AMN is a dynamic network architecture that creates variable numbers of memory banks weighted by question relevance.
Thus, the decoder can select a variable number of memory banks to construct an answer using fewer banks, creating a runtime trade-off between accuracy and speed.
AMN is enabled by first, a novel bank controller that makes discrete decisions with high accuracy and second, the capabilities of a dynamic framework (such as PyTorch) that allow for dynamic network sizing and efficient variable mini-batching.
In our results, we demonstrate that our model learns to construct a varying number of memory banks based on task complexity and achieves faster inference times for standard bAbI tasks, and modified bAbI tasks.
We achieve state of the art accuracy over these tasks with an average 48% lower entities are examined during inference.

assistant: text: Memory networks with faster inference
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Modulating transfer between tasks in gradient-based meta-learning

Paper content:
Learning-to-learn or meta-learning leverages data-driven inductive bias to increase the efficiency of learning on a novel task.
This approach encounters difficulty when transfer is not mutually beneficial, for instance, when tasks are sufficiently dissimilar or change over time.
Here, we use the connection between gradient-based meta-learning and hierarchical Bayes to propose a mixture of hierarchical Bayesian models over the parameters of an arbitrary function approximator such as a neural network.
Generalizing the model-agnostic meta-learning (MAML) algorithm, we present a stochastic expectation maximization procedure to jointly estimate parameter initializations for gradient descent as well as a latent assignment of tasks to initializations.
This approach better captures the diversity of training tasks as opposed to consolidating inductive biases into a single set of hyperparameters.
Our experiments demonstrate better generalization on the standard miniImageNet benchmark for 1-shot classification.
We further derive a novel and scalable non-parametric variant of our method that captures the evolution of a task distribution over time as demonstrated on a set of few-shot regression tasks.

assistant: text: We use the connection between gradient-based meta-learning and hierarchical Bayes to learn a mixture of meta-learners that is appropriate for a heterogeneous and evolving task distribution.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: TCAV: Relative concept importance testing with Linear Concept Activation Vectors

Paper content:
Despite neural network’s high performance, the lack of interpretability has been the main bottleneck for its safe usage in practice.
In domains with high stakes (e.g., medical diagnosis), gaining insights into the network is critical for gaining trust and being adopted.
One of the ways to improve interpretability of a NN is to explain the importance of a particular concept (e.g., gender) in prediction.
This is useful for explaining reasoning behind the networks’ predictions, and for revealing any biases the network may have.
This work aims to provide quantitative answers to \textit{the relative importance of concepts of interest} via concept activation vectors (CAV).
In particular, this framework enables non-machine learning experts to express concepts of interests and  test hypotheses using examples (e.g., a set of pictures that illustrate  the concept).
We show that CAV can be learned given a relatively small set of examples.
Testing with CAV, for example, can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other set of concepts.
Interpreting with CAV does not require any retraining or modification of the network.
We show that many levels of meaningful concepts are learned (e.g., color, texture, objects, a person’s occupation), and we present CAV’s \textit{empirical deepdream} — where we maximize an activation using a set of example pictures.
We show how various insights can be gained from the relative importance testing with CAV.

assistant: text: This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Bijectors.jl: Flexible transformations for probability distributions

Paper content:
Transforming one probability distribution to another is a powerful tool in Bayesian inference and machine learning.
Some prominent examples are constrained-to-unconstrained transformations of distributions for use in Hamiltonian Monte-Carlo and constructing flexible and learnable densities such as normalizing flows.
We present Bijectors.jl, a software package for transforming distributions implemented in Julia, available at github.com/TuringLang/Bijectors.jl.
The package provides a flexible and composable way of implementing transformations of distributions without being tied to a computational framework.
We demonstrate the use of Bijectors.jl on improving variational inference by encoding known statistical dependencies into the variational posterior using normalizing flows, providing a general approach to relaxing the mean-field assumption usually made in variational inference.

assistant: text: We present a software framework for transforming distributions and demonstrate its flexibility on relaxing mean-field assumptions in variational inference with the use of coupling flows to replicate structure from the target generative model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adaptive Mixture of Low-Rank Factorizations for Compact Neural Modeling

Paper content:
Modern deep neural networks have a large amount of weights, which make them difficult to deploy on computation constrained devices such as mobile phones.
One common approach to reduce the model size and computational cost is to use low-rank factorization to approximate a weight matrix.
However, performing standard low-rank factorization with a small rank can hurt the model expressiveness and significantly decrease the performance.
In this work, we propose to use a mixture of multiple low-rank factorizations to model a large weight matrix, and the mixture coefficients are computed dynamically depending on its input.
We demonstrate the effectiveness of the proposed approach on both language modeling and image classification tasks.
Experiments show that our method not only improves the computation efficiency but also maintains (sometimes outperforms) its accuracy compared with the full-rank counterparts.

assistant: text: A simple modification to low-rank factorization that improves performances (in both image and language tasks) while still being compact.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Hyperbolic Image Embeddings

Paper content:
Computer vision tasks such as image classification, image retrieval and few-shot learning are currently dominated by Euclidean and spherical embeddings, so that the final decisions about class belongings or the degree of similarity are made using linear hyperplanes, Euclidean distances, or spherical geodesic distances (cosine similarity).
In this work, we demonstrate that in many practical scenarios hyperbolic embeddings provide a better alternative.

assistant: text: We show that hyperbolic embeddings are useful for high-level computer vision tasks, especially for few-shot classification.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Unsupervised Distillation of Syntactic Information from Contextualized Word Representations

Paper content:
Contextualized word representations, such as ELMo and BERT, were shown to perform well on a various of semantic and structural (syntactic) task.
In this work, we tackle the task of unsupervised disentanglement between semantics and structure in neural language representations: we aim to learn a transformation of the contextualized vectors, that discards the lexical semantics, but keeps the structural information.
To this end, we automatically generate groups of sentences which are structurally similar but semantically different, and use metric-learning approach to learn a transformation that emphasizes the structural component that is encoded in the vectors.
We demonstrate that our transformation clusters vectors in space by structural properties, rather than by lexical semantics.
Finally, we demonstrate the utility of our distilled representations by showing that they outperform the original contextualized representations in few-shot parsing setting.

assistant: text: We distill language models representations for syntax by unsupervised metric learning
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Text Embedding Bank Module for Detailed Image Paragraph Caption

Paper content:
Image paragraph captioning is the task of automatically generating multiple sentences for describing images in grain-fined and coherent text.
Existing typical deep learning-based models for image captioning consist of an image encoder to extract visual features and a language model decoder, which has shown promising results in single high-level sentence generation.
However, only the word-level scalar guiding signal is available when the image encoder is optimized to extract visual features.
The inconsistency between the parallel extraction of visual features and sequential text supervision limits its success when the length of the generated text is long (more than 50 words).
In this paper, we propose a new module, called the Text Embedding Bank (TEB) module, to address the problem for image paragraph captioning.
This module uses the paragraph vector model to learn fixed-length feature representations from a variable-length paragraph.
We refer to the fixed-length feature as the TEB.
This TEB module plays two roles to benefit paragraph captioning performance.
First, it acts as a form of global and coherent deep supervision to regularize visual feature extraction in the image encoder.
Second, it acts as a distributed memory to provide features of the whole paragraph to the language model, which alleviating the long-term dependency problem.
Adding this module to two existing state-of-the-art methods achieves a new state-of-the-art result by a large margin on the paragraph captioning Visual Genome dataset.

assistant: text: TEB Module for IPC
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Discovery of Natural Language Concepts in Individual Units of CNNs

Paper content:
Although deep convolutional networks have achieved improved performance in many natural language tasks, they have been treated as black boxes because they are difficult to interpret.
Especially, little is known about how they represent language in their intermediate layers.
In an attempt to understand the representations of deep convolutional networks trained on language tasks, we show that individual units are selectively responsive to specific morphemes, words, and phrases, rather than responding to arbitrary and uninterpretable patterns.
In order to quantitatively analyze such intriguing phenomenon, we propose a concept alignment method based on how units respond to replicated text.
We conduct analyses with different architectures on multiple datasets for classification and translation tasks and provide new insights into how deep models understand natural language.

assistant: text: We show that individual units in CNN representations learned in NLP tasks are selectively responsive to natural language concepts.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Detecting Anomalies in Communication Packet Streams based on  Generative Adversarial Networks

Paper content:
The fault diagnosis in a modern communication system is traditionally supposed to be difficult, or even impractical for a purely data-driven machine learning approach, for it is a humanmade system of intensive knowledge.
A few labeled raw packet streams extracted from fault archive can hardly be sufficient to deduce the intricate logic of underlying protocols.
In this paper, we supplement these limited samples with two inexhaustible data sources: the unlabeled records probed from a system in service, and the labeled data simulated in an emulation environment.
To transfer their inherent knowledge to the target domain, we construct a directed information flow graph, whose nodes are neural network components consisting of two generators, three discriminators and one classifier, and whose every forward path represents a pair of adversarial optimization goals, in accord with the semi-supervised and transfer learning demands.
The multi-headed network can be trained in an alternative approach, at each iteration of which we select one target to update the weights along the path upstream, and refresh the residual layer-wisely to all outputs downstream.
The actual results show that it can achieve comparable accuracy on classifying Transmission Control Protocol (TCP) streams without deliberate expert features.
The solution has relieved operation engineers from massive works of understanding and maintaining rules, and provided a quick solution independent of specific protocols.

assistant: text: semi-supervised and transfer learning on packet flow classification, via a system of cooperative or adversarial neural blocks
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
This paper introduces an information theoretic co-training objective for unsupervised learning.
We consider the problem of predicting the future.
Rather than predict future sensations (image pixels or sound waves) we predict ``hypotheses'' to be confirmed by future sensations.
More formally, we assume a population distribution on pairs $(x,y)$ where we can think of $x$ as a past sensation and $y$ as a future sensation.
We train both a predictor model $P_\Phi(z|x)$ and a confirmation model $P_\Psi(z|y)$ where we view $z$ as hypotheses (when predicted) or facts (when confirmed).
For a population distribution on pairs $(x,y)$ we focus on the problem of measuring the mutual information between $x$ and $y$. By the data processing inequality this mutual information is at least as large as the mutual information between $x$ and $z$ under the distribution on triples $(x,z,y)$ defined by the confirmation model $P_\Psi(z|y)$. The information theoretic training objective for $P_\Phi(z|x)$ and $P_\Psi(z|y)$ can be viewed as a form of co-training where we want the prediction from $x$ to match the confirmation from $y$. We give experiments on applications to learning phonetics on the TIMIT dataset.

assistant: text: Presents an information theoretic training objective for co-training and demonstrates its power in unsupervised learning of phonetics.