ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Size-free generalization bounds for convolutional neural networks

Paper content:
We prove bounds on the generalization error of convolutional networks.
The bounds are in terms of the training loss, the number of
parameters, the Lipschitz constant of the loss and the distance from
the weights to the initial weights.
They are independent of the
number of pixels in the input, and the height and width of hidden
feature maps.
We present experiments with CIFAR-10, along with varying
hyperparameters of a deep convolutional network, comparing our bounds
with practical generalization gaps.

assistant: text: We prove generalization bounds for convolutional neural networks that take account of weight-tying
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Robust Few-Shot Learning with Adversarially Queried Meta-Learners

Paper content:
Previous work on adversarially robust neural networks requires large training sets and computationally expensive training procedures.
On the other hand, few-shot learning methods are highly vulnerable to adversarial examples.
The goal of our work is to produce networks which both perform well at few-shot tasks and are simultaneously robust to adversarial examples.
We adapt adversarial training for meta-learning, we adapt robust architectural features to small networks for meta-learning, we test pre-processing defenses as an alternative to adversarial training for meta-learning, and we investigate the advantages of robust meta-learning over robust transfer-learning for few-shot tasks.
This work provides a thorough analysis of adversarially robust methods in the context of meta-learning, and we lay the foundation for future work on defenses for few-shot tasks.

assistant: text: This paper presents a method that enhances the robustness of few-shot learning by introducing adversarial query data attack in the inner-task fine-tuning phase of a meta-learning algorithm.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Melody Generation for Pop Music via Word Representation of Musical Properties

Paper content:
Automatic melody generation for pop music has been a long-time aspiration for
both AI researchers and musicians.
However, learning to generate euphonious
melody has turned out to be highly challenging due to a number of factors.
Representation
of multivariate property of notes has been one of the primary challenges.
It is also difficult to remain in the permissible spectrum of musical variety, outside
of which would be perceived as a plain random play without auditory pleasantness.
Observing the conventional structure of pop music poses further challenges.
In this paper, we propose to represent each note and its properties as a unique
‘word,’ thus lessening the prospect of misalignments between the properties, as
well as reducing the complexity of learning.
We also enforce regularization policies
on the range of notes, thus encouraging the generated melody to stay close
to what humans would find easy to follow.
Furthermore, we generate melody
conditioned on song part information, thus replicating the overall structure of a
full song.
Experimental results demonstrate that our model can generate auditorily
pleasant songs that are more indistinguishable from human-written ones than
previous models.

assistant: text: This paper proposes a generative model of symbolic (MIDI) melody in western popular music which jointly encodes note symbols with timing and duration information to form musical "words".
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification

Paper content:
Modern deep neural networks (DNNs) require high memory consumption and large computational loads.
In order to deploy DNN algorithms efficiently on edge or mobile devices, a series of DNN compression algorithms have been explored, including the line of works on factorization methods.
Factorization methods approximate the weight matrix of a DNN layer with multiplication of two or multiple low-rank matrices.
However, it is hard to measure the ranks of DNN layers during the training process.
Previous works mainly induce low-rank through implicit approximations or via costly singular value decomposition (SVD) process on every training step.
The former approach usually induces a high accuracy loss while the latter prevents DNN factorization from efficiently reaching a high compression rate.
In this work, we propose SVD training, which first applies SVD to decompose DNN's layers and then performs training on the full-rank decomposed weights.
To improve the training quality and convergence, we add orthogonality regularization to the singular vectors, which ensure the valid form of SVD and avoid gradient vanishing/exploding.
Low-rank is encouraged by applying sparsity-inducing regularizers on the singular values of each layer.
Singular value pruning is applied at the end to reach a low-rank model.
We empirically show that SVD training can significantly reduce the rank of DNN layers and achieve higher reduction on computation load under the same accuracy, comparing to not only previous factorization methods but also state-of-the-art filter pruning methods.

assistant: text: This paper introduces an approach to network compression by encouraging the weight matrix in each layer to have a low rank and explicitly factorizing the weight matrices into an SVD-like factorization for treatment as new parameters.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Scoring-Aggregating-Planning: Learning task-agnostic priors from interactions and sparse rewards for zero-shot generalization

Paper content:
Humans can learn task-agnostic priors from interactive experience and utilize the priors for novel tasks without any finetuning.
In this paper, we propose Scoring-Aggregating-Planning (SAP), a framework that can learn task-agnostic semantics and dynamics priors from arbitrary quality interactions as well as the corresponding sparse rewards and then plan on unseen tasks in zero-shot condition.
The framework finds a neural score function for local regional state and action pairs that can be aggregated to approximate the quality of a full trajectory; moreover, a dynamics model that is learned with self-supervision can be incorporated for planning.
Many of previous works that leverage interactive data for policy learning either need massive on-policy environmental interactions or assume access to expert data while we can achieve a similar goal with pure off-policy imperfect data.
Instantiating our framework results in a generalizable policy to unseen tasks.
Experiments demonstrate that the proposed method can outperform baseline methods on a wide range of applications including gridworld, robotics tasks and video games.

assistant: text: This paper discusses zero shot generalization into new environments, and proposes an approach with results on Grid-World, Super Mario Bros, and 3D Robotics.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Few-Shot Regression via Learned Basis Functions

Paper content:
The recent rise in popularity of few-shot learning algorithms has enabled models to quickly adapt to new tasks based on only a few training samples.
Previous few-shot learning works have mainly focused on classification and reinforcement learning.
In this paper, we propose a few-shot meta-learning system that focuses exclusively on regression tasks.
Our model is based on the idea that the degree of freedom of the unknown function can be significantly reduced if it is represented as a linear combination of a set of appropriate basis functions.
This enables a few labelled samples to approximate the function.
We design a Feature Extractor network to encode basis functions for a task distribution, and a  Weights Generator to generate the weight vector for a novel task.
We show that our model outperforms the current state of the art meta-learning methods in various regression tasks.

assistant: text: A method that learns a regression model with a few samples and outperforms other methods.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: TimbreTron: A WaveNet(CycleGAN(CQT(Audio))) Pipeline for Musical Timbre Transfer

Paper content:
In this work, we address the problem of musical timbre transfer, where the goal is to manipulate the timbre of a sound sample from one instrument to match another instrument while preserving other musical content, such as pitch, rhythm, and loudness.
In principle, one could apply image-based style transfer techniques to a time-frequency representation of an audio signal, but this depends on having a representation that allows independent manipulation of timbre as well as high-quality waveform generation.
We introduce TimbreTron, a method for musical timbre transfer which applies “image” domain style transfer to a time-frequency representation of the audio signal, and then produces a high-quality waveform using a conditional WaveNet synthesizer.
We show that the Constant Q Transform (CQT) representation is particularly well-suited to convolutional architectures due to its approximate pitch equivariance.
Based on human perceptual evaluations, we confirmed that TimbreTron recognizably transferred the timbre while otherwise preserving the musical content, for both monophonic and polyphonic samples.
We made an accompanying demo video here: https://www.cs.toronto.edu/~huang/TimbreTron/index.html which we strongly encourage you to watch before reading the paper.

assistant: text: A method for converting recordings of a specific musical instrument to another by applying CycleGAN, developed for image style transfer, to transfer spectrograms.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: TFGAN: Improving Conditioning for Text-to-Video Synthesis

Paper content:
Developing conditional generative models for text-to-video synthesis is an extremely challenging yet an important topic of research in machine learning.
In this work, we address this problem by introducing Text-Filter conditioning Generative Adversarial Network (TFGAN), a GAN model with novel conditioning scheme that aids improving the text-video associations.
With a combination of this conditioning scheme and a deep GAN architecture, TFGAN generates photo-realistic videos from text on very challenging real-world video datasets.
In addition, we construct a benchmark synthetic dataset of moving shapes to systematically evaluate our conditioning scheme.
Extensive experiments demonstrate that TFGAN significantly outperforms the existing approaches, and can also generate videos of novel categories not seen during training.

assistant: text: This paper proposes conditional GAN models for text-to-video synthesis: developing text-feature-conditioned CNN filters and constructing moving-shape dataset with improved performance on video/image generation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient Inference and Exploration for Reinforcement Learning

Paper content:
Despite an ever growing literature on reinforcement learning algorithms and applications, much less is known about their statistical inference.
In this paper, we investigate the large-sample behaviors of the Q-value estimates with closed-form characterizations of the asymptotic variances.
This allows us to efficiently construct confidence regions for Q-value and optimal value functions, and to develop policies to minimize their estimation errors.
This also leads to a policy exploration strategy that relies on estimating the relative discrepancies among the Q estimates.
Numerical experiments show superior performances of our exploration strategy than other benchmark approaches.

assistant: text: This paper presents a pure-exploration algorithm for reinforcement learning based on an asymptotic analysis of Q-values and their convergence to central limit distribution, outperforming benchmark exploration algorithms.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On Meaning-Preserving Adversarial Perturbations for Sequence-to-Sequence Models

Paper content:
Adversarial examples have been shown to be an effective way of assessing the robustness of neural sequence-to-sequence (seq2seq) models, by applying perturbations to the input of a model leading to large degradation in performance.
However, these perturbations are only indicative of a weakness in the model if they do not change the semantics of the input in a way that would change the expected output.
Using the example of machine translation (MT), we propose a new evaluation framework for adversarial attacks on seq2seq models taking meaning preservation into account and demonstrate that existing methods may not preserve meaning in general.
Based on these findings, we propose new constraints for attacks on word-based MT systems and show, via human and automatic evaluation, that they produce more semantically similar adversarial inputs.
Furthermore, we show that performing adversarial training with meaning-preserving attacks is beneficial to the model in terms of adversarial robustness without hurting test performance.

assistant: text: The authors investigate ways of generating adversarial examples, showing that adversarial training with the attack most consistent with the introduced meaning-preservation criteria results in improved robustness to this type of attack without degradation in the non-adversarial setting.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step

Paper content:
Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion.
Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost.
GANs are designed to reach a Nash equilibrium at which each player cannot reduce their cost without changing the other players’ parameters.
One useful approach for the theory of GANs is to show that a divergence between the training distribution and the model distribution obtains its minimum value at equilibrium.
Several recent research directions have been motivated by the idea that this divergence is the primary guide for the learning process and that every step of learning should decrease the divergence.
We show that this view is overly restrictive.
During GAN training, the discriminator provides learning signal in situations where the gradients of the divergences between distributions would not be useful.
We provide empirical counterexamples to the view of GAN training as divergence minimization.
Specifically, we demonstrate that GANs are able to learn distributions in situations where the divergence minimization point of view predicts they would fail.
We also show that gradient penalties motivated from the divergence minimization perspective are equally helpful when applied in other contexts in which the divergence minimization perspective does not predict they would be helpful.
This contributes to a growing body of evidence that GAN training may be more usefully viewed as approaching Nash equilibria via trajectories that do not necessarily minimize a specific divergence at each step.

assistant: text: The submission aims to present empirical evidence that the theory of divergence minimization is more a tool to understand the outcome of training GANs than a necessary condition to be enforce during training itself
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: LSH-SAMPLING BREAKS THE COMPUTATIONAL CHICKEN-AND-EGG LOOP IN ADAPTIVE STOCHASTIC GRADIENT ESTIMATION

Paper content:
Stochastic Gradient Descent or SGD is the most popular optimization algorithm for large-scale problems.
SGD estimates the gradient by uniform sampling with sample size one.
There have been several other works that suggest faster epoch wise convergence by using weighted non-uniform sampling for better gradient estimates.
Unfortunately, the per-iteration cost of maintaining this adaptive distribution for gradient estimation is more than calculating the full gradient.
As a result, the false impression of faster convergence in iterations leads to slower convergence in time, which we call a chicken-and-egg loop.
In this paper, we break this barrier by providing the first demonstration of a sampling scheme, which leads to superior gradient estimation, while keeping the sampling cost per iteration similar to that of the uniform sampling.
Such an algorithm is possible due to the sampling view of Locality Sensitive Hashing (LSH), which came to light recently.
As a consequence of superior and fast estimation, we reduce the running time of all existing gradient descent algorithms.
We demonstrate the benefits of our proposal on both SGD and AdaGrad.

assistant: text: Considers SGD over an objective of the form of a sum over examples of a quadratic loss.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Top-Down Neural Model For Formulae

Paper content:
We present a simple neural model that given a formula and a property tries to answer the question whether the formula has the given property, for example whether a propositional formula is always true.
The structure of the formula is captured by a feedforward neural network recursively built for the given formula in a top-down manner.
The results of this network are then processed by two recurrent neural networks.
One of the interesting aspects of our model is how propositional atoms are treated.
For example, the model is insensitive to their names, it only matters whether they are the same or distinct.

assistant: text: The paper pursues the path of a tree-structured network isomorphic to the parse tree of a propositional-calculus formula, but by passing information top-down rather than bottom-up.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On the Global Convergence  of Training Deep Linear ResNets

Paper content:
We study the convergence of gradient descent (GD) and stochastic gradient descent (SGD) for training $L$-hidden-layer linear residual networks (ResNets).
We prove that for training deep residual networks with certain linear transformations at input and output layers, which are fixed throughout training, both GD and SGD with zero initialization on all hidden weights can converge to the global minimum of the training loss.
Moreover, when specializing to appropriate Gaussian random linear transformations, GD and SGD provably optimize wide enough deep linear ResNets.
Compared with the global convergence result of GD for training standard deep linear networks \citep{du2019width}, our condition on the neural network width is sharper by a factor of $O(\kappa L)$, where $\kappa$ denotes the condition number of the covariance matrix of the training data.
In addition, for the first time we establish the global convergence of SGD for training deep linear ResNets and prove a linear convergence rate when the global minimum is $0$.

assistant: text: The authors study the convergence of gradient descent in training deep linear residual networks, and establish a global convergence of GD/SGD and linear convergence rates of SG/SGD.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Bayesian Inference for Large Scale Image Classification

Paper content:
Bayesian inference promises to ground and improve the performance of deep neural networks.
It promises to be robust to overfitting, to simplify the training procedure and the space of hyperparameters, and to provide a calibrated measure of uncertainty that can enhance decision making, agent exploration and prediction fairness.
Markov Chain Monte Carlo (MCMC) methods enable Bayesian inference by generating samples from the posterior distribution over model parameters.
Despite the theoretical advantages of Bayesian inference and the similarity between MCMC and optimization methods, the performance of sampling methods has so far lagged behind  optimization methods for large scale deep learning tasks.
We aim to fill this gap and introduce ATMC, an adaptive noise MCMC algorithm that estimates and is able to sample from the posterior of a neural network.
ATMC dynamically adjusts the amount of momentum and noise applied to each parameter update in order to compensate for the use of stochastic gradients.
We use a ResNet architecture without batch normalization to test ATMC on the Cifar10 benchmark and the large scale ImageNet benchmark and show that, despite the  absence of batch normalization, ATMC outperforms a strong optimization baseline in terms of both classification accuracy and test log-likelihood.
We show that ATMC is intrinsically robust to overfitting on the training data and that ATMC provides a better calibrated measure of uncertainty compared to the optimization baseline.

assistant: text: An adaptive noise MCMC algorithm for image classification that dynamically adjusts the momentum and noise applied to each parameter update, and is robust to overfitting and provides an uncertainty measure with predictions.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Universality Theorems for Generative Models

Paper content:
Despite the fact that generative models are extremely successful in practice, the theory underlying this phenomenon is only starting to catch up with practice.
In this work we address the question of the universality of generative models: is it true that neural networks can approximate any data manifold arbitrarily well?
We provide a positive answer to this question and show that under mild assumptions on the activation function one can always find a feedforward neural network that maps the latent space onto a set located within the specified Hausdorff distance from the desired data manifold.
We also prove similar theorems for the case of multiclass generative models and cycle generative models, trained to map samples from one manifold to another and vice versa.

assistant: text: This paper theoretically shows that neural-network-based generative models can approximate data manifolds, and proves that under mild assumptions neural networks can map a latent space onto a set close to the given data manifold within a small Hausdorff distance.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep Perm-Set Net: Learn to predict sets with unknown permutation and cardinality using deep neural networks

Paper content:
Many real-world problems, e.g. object detection, have outputs that are naturally expressed as sets of entities.
This creates a challenge for traditional deep neural networks which naturally deal with structured outputs such as vectors, matrices or tensors.
We present a novel approach for learning to predict sets with unknown permutation and cardinality using deep neural networks.
Specifically, in our formulation we incorporate the permutation as unobservable variable and estimate its distribution during the learning process using alternating optimization.
We demonstrate the validity of this new formulation on two relevant vision problems: object detection, for which our formulation outperforms state-of-the-art detectors such as Faster R-CNN and YOLO, and a complex CAPTCHA test, where we observe that, surprisingly, our set based network acquired the ability of mimicking arithmetics without any rules being coded.

assistant: text: A formulation to learn the distribution over unobservable permutation variables based on deep networks for the set prediction problem.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Challenges of Explaining Control

Paper content:
Reinforcement learning and evolutionary algorithms can be used to create sophisticated control solutions.
Unfortunately explaining how these solutions work can be difficult to due to their "black box" nature.
In addition, the time-extended nature of control algorithms often prevent direct applications of explainability techniques used for standard supervised learning algorithms.
This paper attempts to address explainability of blackbox control algorithms through six different techniques:
1) Bayesian rule lists,
2) Function analysis,
3) Single time step integrated gradients,
4) Grammar-based decision trees,
5) Sensitivity analysis combined with temporal modeling with LSTMs, and
6) Explanation templates.
These techniques are tested on a simple 2d domain, where a simulated rover attempts to navigate through obstacles to reach a goal.
For control, this rover uses an evolved multi-layer perception that maps an 8d field of obstacle and goal sensors to an action determining where it should go in the next time step.
Results show that some simple insights in explaining the neural network are possible, but that good explanations are difficult.

assistant: text: This paper provides insights and explanations for the problem of providing explanations for a multilayer perceptron used as an inverse controller for rover movement, and ideas on how to explain a black-box model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: BlackMarks: Black-box Multi-bit Watermarking for Deep Neural Networks

Paper content:
Deep Neural Networks (DNNs) are increasingly deployed in cloud servers and autonomous agents due to their superior performance.
The deployed DNN is either leveraged in a white-box setting (model internals are publicly known) or a black-box setting (only model outputs are known) depending on the application.
A practical concern in the rush to adopt DNNs is protecting the models against Intellectual Property (IP) infringement.
We propose BlackMarks, the first end-to-end multi-bit watermarking framework that is applicable in the black-box scenario.
BlackMarks takes the pre-trained unmarked model and the owner’s binary signature as inputs.
The output is the corresponding marked model with specific keys that can be later used to trigger the embedded watermark.
To do so, BlackMarks first designs a model-dependent encoding scheme that maps all possible classes in the task to bit ‘0’ and bit ‘1’.
Given the owner’s watermark signature (a binary string), a set of key image and label pairs is designed using targeted adversarial attacks.
The watermark (WM) is then encoded in the distribution of output activations of the DNN by fine-tuning the model with a WM-specific regularized loss.
To extract the WM, BlackMarks queries the model with the WM key images and decodes the owner’s signature from the corresponding predictions using the designed encoding scheme.
We perform a comprehensive evaluation of BlackMarks’ performance on MNIST, CIFAR-10, ImageNet datasets and corroborate its effectiveness and robustness.
BlackMarks preserves the functionality of the original DNN and incurs negligible WM embedding overhead as low as 2.054%.

assistant: text: Proposes a method for multi-bit watermarking of neural networks in a black-box setting and demonstrate that the predictions of existing models can carry a multi-bit string that can later be used to verify ownership.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Boosting Dilated Convolutional Networks with Mixed Tensor Decompositions

Paper content:
The driving force behind deep networks is their ability to compactly represent rich classes of functions.
The primary notion for formally reasoning about this phenomenon is expressive efficiency, which refers to a situation where one network must grow unfeasibly large in order to replicate functions of another.
To date, expressive efficiency analyses focused on the architectural feature of depth, showing that deep networks are representationally superior to shallow ones.
In this paper we study the expressive efficiency brought forth by connectivity, motivated by the observation that modern networks interconnect their layers in elaborate ways.
We focus on dilated convolutional networks, a family of deep models delivering state of the art performance in sequence processing tasks.
By introducing and analyzing the concept of mixed tensor decompositions, we prove that interconnecting dilated convolutional networks can lead to expressive efficiency.
In particular, we show that even a single connection between intermediate layers can already lead to an almost quadratic gap, which in large-scale settings typically makes the difference between a model that is practical and one that is not.
Empirical evaluation demonstrates how the expressive efficiency of connectivity, similarly to that of depth, translates into gains in accuracy.
This leads us to believe that expressive efficiency may serve a key role in developing new tools for deep network design.

assistant: text: The authors study dilated convolutional networks and show that intertwining two dilated convolutional networks A and B at various stages is more expressively efficient than not intertwining.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Mapping the hyponymy relation of wordnet onto vector Spaces

Paper content:
In this paper, we investigate mapping the hyponymy relation of
 wordnet to feature vectors.
We aim to model lexical knowledge in such a way that it can be used as
  input in generic machine-learning models, such as phrase entailment
  predictors.
We propose two models.
The first one leverages an existing mapping of
  words to feature vectors (fasttext), and attempts to classify
  such vectors as within or outside of each class.
The second model is fully supervised,
  using solely wordnet as a ground truth.
It maps each concept to an
  interval or a disjunction thereof.
On the first model, we approach, but not quite attain state of the
  art performance.
The second model can achieve near-perfect accuracy.

assistant: text: This paper explores the notion of hyponymy in word vector representations and describes a method of organizing WordNet relations into a tree structure to define hyponymy.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures

Paper content:
In seeking for sparse and efficient neural network models, many previous works investigated on enforcing L1 or L0 regularizers to encourage weight sparsity during training.
The L0 regularizer measures the parameter sparsity directly and is invariant to the scaling of parameter values.
But it cannot provide useful gradients and therefore requires complex optimization techniques.
The L1 regularizer is almost everywhere differentiable and can be easily optimized with gradient descent.
Yet it is not scale-invariant and causes the same shrinking rate to all parameters, which is inefficient in increasing sparsity.
Inspired by the Hoyer measure (the ratio between L1 and L2 norms) used in traditional compressed sensing problems, we present DeepHoyer, a set of sparsity-inducing regularizers that are both differentiable almost everywhere and scale-invariant.
Our experiments show that enforcing DeepHoyer regularizers can produce even sparser neural network models than previous works, under the same accuracy level.
We also show that DeepHoyer can be applied to both element-wise and structural pruning.

assistant: text: We propose almost everywhere differentiable and scale invariant regularizers for DNN pruning, which can lead to supremum sparsity through standard SGD training.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Classification-Based Anomaly Detection for General Data

Paper content:
Anomaly detection, finding patterns that substantially deviate from those seen previously, is one of the fundamental problems of artificial intelligence.
Recently, classification-based methods were shown to achieve superior results on this task.
In this work, we present a unifying view and propose an open-set method to relax current generalization assumptions.
Furthermore, we extend the applicability of transformation-based methods to non-image data using random affine transformations.
Our method is shown to obtain state-of-the-art accuracy and is applicable to broad data types.
The strong performance of our method is extensively validated on multiple datasets from different domains.

assistant: text: An anomaly detection that:  uses random-transformation classification for generalizing to non-image data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Semi-supervised 3D Face Reconstruction with Nonlinear Disentangled Representations

Paper content:
Recovering 3D geometry shape, albedo and lighting from a single image has wide applications in many areas, which is also a typical ill-posed problem.
In order to eliminate the ambiguity, face prior knowledge like linear 3D morphable models (3DMM) learned from limited scan data are often adopted to the reconstruction process.
However, methods based on linear parametric models cannot generalize well for facial images in the wild with various ages, ethnicity, expressions, poses, and lightings.
Recent methods aim to learn a nonlinear parametric model using convolutional neural networks (CNN) to regress the face shape and texture directly.
However, the models were only trained on a dataset that is generated from a linear 3DMM.
Moreover, the identity and expression representations are entangled in these models, which hurdles many facial editing applications.
In this paper, we train our model with adversarial loss in a semi-supervised manner on hybrid batches of unlabeled and labeled face images to exploit the value of large amounts of unlabeled face images from unconstrained photo collections.
A novel center loss is introduced to make sure that different facial images from the same person have the same identity shape and albedo.
Besides, our proposed model disentangles identity, expression, pose, and lighting representations, which improves the overall reconstruction performance and facilitates facial editing applications, e.g., expression transfer.
Comprehensive experiments demonstrate that our model produces high-quality reconstruction compared to state-of-the-art methods and is robust to various expression, pose, and lighting conditions.

assistant: text: This paper proposes a semi-supervised and adversarial training process to exact nonlinear disentangled representations from a face image with loss functions, achieving state-of-the-art performance in face reconstruction.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Actionable Representations with Goal Conditioned Policies

Paper content:
Representation learning is a central challenge across a range of machine learning areas.
In reinforcement learning, effective and functional representations have the potential to tremendously accelerate learning progress and solve more challenging problems.
Most prior work on representation learning has focused on generative approaches, learning representations that capture all the underlying factors of variation in the observation space in a more disentangled or well-ordered manner.
In this paper, we instead aim to learn functionally salient representations: representations that are not necessarily complete in terms of capturing all factors of variation in the observation space, but rather aim to capture those factors of variation that are important for decision making -- that are "actionable".
These representations are aware of the dynamics of the environment, and capture only the elements of the observation that are necessary for decision making rather than all factors of variation, eliminating the need for explicit reconstruction.
We show how these learned representations can be useful to improve exploration for sparse reward problems, to enable long horizon hierarchical reinforcement learning, and as a state representation for learning policies for downstream tasks.
We evaluate our method on a number of simulated environments, and compare it to prior methods for representation learning, exploration, and hierarchical reinforcement learning.

assistant: text: The paper presents a method to learn representations where proximity in euclidean distance represents states that are achieved by similar policies.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Classification-Based Anomaly Detection for General Data

Paper content:
Anomaly detection, finding patterns that substantially deviate from those seen previously, is one of the fundamental problems of artificial intelligence.
Recently, classification-based methods were shown to achieve superior results on this task.
In this work, we present a unifying view and propose an open-set method to relax current generalization assumptions.
Furthermore, we extend the applicability of transformation-based methods to non-image data using random affine transformations.
Our method is shown to obtain state-of-the-art accuracy and is applicable to broad data types.
The strong performance of our method is extensively validated on multiple datasets from different domains.

assistant: text: This paper proposes a deep method for anomaly detection that unifies recent deep one-class classification and transformation-based classification approaches.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Can I trust you more? Model-Agnostic Hierarchical Explanations

Paper content:
Interactions such as double negation in sentences and scene interactions in images are common forms of complex dependencies captured by state-of-the-art machine learning models.
We propose Mahé, a novel approach to provide Model-Agnostic Hierarchical Explanations of how powerful machine learning models, such as deep neural networks, capture these interactions as either dependent on or free of the context of data instances.
Specifically, Mahé provides context-dependent explanations by a novel local interpretation algorithm that effectively captures any-order interactions, and obtains context-free explanations through generalizing context-dependent interactions to explain global behaviors.
Experimental results show that Mahé obtains improved local interaction interpretations over state-of-the-art methods and successfully provides explanations of interactions that are context-free.

assistant: text: A method that can provide hierarchical explanations for a model, including both context-dependent and context-free explanations by a local interpretation algorithm.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Context Mover's Distance & Barycenters: Optimal transport of contexts for building representations

Paper content:
We present a framework for building unsupervised representations of entities and their compositions, where each entity is viewed as a probability distribution rather than a fixed length vector.
In particular, this distribution is supported over the contexts which co-occur with the entity and are embedded in a suitable low-dimensional space.
This enables us to consider the problem of representation learning with a perspective from Optimal Transport and take advantage of its numerous tools such as Wasserstein distance and Wasserstein barycenters.
We elaborate how the method can be applied for obtaining unsupervised representations of text and illustrate the performance quantitatively as well as qualitatively on tasks such as measuring sentence similarity and word entailment, where we empirically observe significant gains (e.g., 4.1% relative improvement over Sent2vec and GenSen).
The key benefits of the proposed approach include:
(a) capturing uncertainty and polysemy via modeling the entities as distributions,
(b) utilizing the underlying geometry of the particular task (with the ground cost),
(c) simultaneously providing interpretability with the notion of optimal transport between contexts and
(d) easy applicability on top of existing point embedding methods.
In essence, the framework can be useful for any unsupervised or supervised problem (on text or other modalities); and only requires a co-occurrence structure inherent to many problems.
The code, as well as pre-built histograms, are available under https://github.com/context-mover.

assistant: text: Proposes to construct word embeddings from a histogram over context words, instead of as point vectors, which allows for measuring distances between two words in terms of optimal transport between the histograms through a method that augments representation of an entity from standard "point in a vector space" to a histogram with bins located at some points in that vector space.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Conversation Generation with Concept Flow

Paper content:
Human conversations naturally evolve around related entities and connected concepts, while may also shift from topic to topic.
This paper presents ConceptFlow, which leverages commonsense knowledge graphs to explicitly model such conversation flows for better conversation response generation.
ConceptFlow grounds the conversation inputs to the latent concept space and represents the potential conversation flow as a concept flow along the commonsense relations.
The concept is guided by a graph attention mechanism that models the possibility of the conversation evolving towards different concepts.
The conversation response is then decoded using the encodings of both utterance texts and concept flows, integrating the learned conversation structure in the concept space.
Our experiments on Reddit conversations demonstrate the advantage of ConceptFlow over previous commonsense aware dialog models and fine-tuned GPT-2 models, while using much fewer parameters but with explicit modeling of conversation structures.

assistant: text: The paper proposes a system for generating a single-turn response to a posted utterance in an open-domain dialog setting using the diffiusion into the neighbors of the grounded concepts.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms

Paper content:
We propose to use a meta-learning objective that maximizes the speed of transfer on a modified distribution to learn how to modularize acquired knowledge.
In particular, we focus on how to factor a joint distribution into appropriate conditionals, consistent with the causal directions.
We explain when this can work, using the assumption that the changes in distributions are localized (e.g. to one of the marginals, for example due to an intervention on one of the variables).
We prove that under this assumption of localized changes in causal mechanisms, the correct causal graph will tend to have only a few of its parameters with non-zero gradient, i.e. that need to be adapted (those of the modified variables).
We argue and observe experimentally that this leads to faster adaptation, and use this property to define a meta-learning surrogate score which, in addition to a continuous parametrization of graphs, would favour correct causal graphs.
Finally, motivated by the AI agent point of view (e.g. of a robot discovering its environment autonomously), we consider how the same objective can discover the causal variables themselves, as a transformation of observed low-level variables with no causal meaning.
Experiments in the two-variable case validate the proposed ideas and theoretical results.

assistant: text: In this work, the authors proposed a general and systematic framework of meta-transfer objective incorporating the causal structure learning under unknown interventions.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Continuous Meta-Learning without Tasks

Paper content:
Meta-learning is a promising strategy for learning to efficiently learn within new tasks, using data gathered from a distribution of tasks.
However, the meta-learning literature thus far has focused on the task segmented setting, where at train-time, offline data is assumed to be split according to the underlying task, and at test-time, the algorithms are optimized to learn in a single task.
In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task.
We present meta-learning via online changepoint analysis (MOCA), an approach which augments a meta-learning algorithm with a differentiable Bayesian changepoint detection scheme.
The framework allows both training and testing directly on time series data without segmenting it into discrete tasks.
We demonstrate the utility of this approach on a nonlinear meta-regression benchmark as well as two meta-image-classification benchmarks.

assistant: text: This paper pushes meta-learning towards task-unsegmented settings, where the MOCA framework adopts a Bayesian changepoint estimation scheme for task change detection.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search

Paper content:
Differentiable architecture search (DARTS) provided a fast solution in finding effective network architectures, but suffered from large memory and computing overheads in jointly training a super-net and searching for an optimal architecture.
In this paper, we present a novel approach, namely  Partially-Connected DARTS, by sampling a small part of super-net to reduce the redundancy in exploring the network space, thereby performing a more efficient search without comprising the performance.
In particular, we perform operation search in a subset of channels while bypassing the held out part in a shortcut.
This strategy may suffer from an undesired inconsistency on selecting the edges of super-net caused by sampling different channels.
We solve it by introducing  edge normalization, which adds a new set of edge-level hyper-parameters to reduce uncertainty in search.
Thanks to the reduced memory cost, PC-DARTS can be trained with a larger batch size and, consequently, enjoy both faster speed and higher training stability.
Experiment results demonstrate the effectiveness of the proposed method.
Specifically, we achieve an error rate of 2.57% on CIFAR10 within merely 0.1 GPU-days for architecture search, and a state-of-the-art top-1 error rate of 24.2% on ImageNet (under the mobile setting) within 3.8 GPU-days for search.
Our code has been made available at https://www.dropbox.com/sh/on9lg3rpx1r6dkf/AABG5mt0sMHjnEJyoRnLEYW4a?dl=0
.

assistant: text: Allowing partial channel connection in super-networks to regularize and accelerate differentiable architecture search