ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Sensitivity of Deep Convolutional Networks to Gabor Noise

Paper content:
Deep Convolutional Networks (DCNs) have been shown to be sensitive to Universal Adversarial Perturbations (UAPs): input-agnostic perturbations that fool a model on large portions of a dataset.
These UAPs exhibit interesting visual patterns, but this phenomena is, as yet, poorly understood.
Our work shows that visually similar procedural noise patterns also act as UAPs.
In particular, we demonstrate that different DCN architectures are sensitive to Gabor noise patterns.
This behaviour, its causes, and implications deserve further in-depth study.

assistant: text: Existing Deep Convolutional Networks in image classification tasks are sensitive to Gabor noise patterns, i.e. small structured changes to the input cause large changes to the output.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control

Paper content:
We propose a "plan online and learn offline" framework for the setting where an agent, with an internal model, needs to continually act and learn in the world.
Our work builds on the synergistic relationship between local model-based control, global value function learning, and exploration.
We study how local trajectory optimization can cope with approximation errors  in the value function, and can stabilize and accelerate value function learning.
Conversely, we also study how approximate value functions can help reduce the planning horizon and allow for better policies beyond local solutions.
Finally, we also demonstrate how trajectory optimization can be used to perform temporally coordinated exploration in conjunction with estimating uncertainty in value function approximation.
This exploration is critical for fast and stable learning of the value function.
Combining these components enable solutions to complex control tasks, like humanoid locomotion and dexterous in-hand manipulation, in the equivalent of a few minutes of experience in the real world.

assistant: text: We propose a framework that incorporates planning for efficient exploration and learning in complex environments.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Formal Limitations on the Measurement of Mutual Information

Paper content:
Motivated by applications to unsupervised learning, we consider the problem of measuring mutual information.
Recent analysis has shown that naive kNN estimators of mutual information have serious statistical limitations motivating more refined methods.
In this paper we prove that serious statistical limitations are inherent to any measurement method.
More specifically, we show that any distribution-free high-confidence lower bound on mutual information cannot be larger than $O(\ln N)$ where $N$ is the size of the data sample.
We also analyze the Donsker-Varadhan lower bound on KL divergence in particular and show that, when simple statistical considerations are taken into account, this bound can never produce a high-confidence value larger than $\ln N$.
While large high-confidence lower bounds are impossible, in practice one can use estimators without formal guarantees.
We suggest expressing mutual information as a difference of entropies and using cross entropy as an entropy estimator.
We observe that, although cross entropy is only an upper bound on entropy, cross-entropy estimates converge to the true cross entropy at the rate of $1/\sqrt{N}$.

assistant: text: We give a theoretical analysis of the measurement and optimization of mutual information.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Single Deep Counterfactual Regret Minimization

Paper content:
Counterfactual Regret Minimization (CFR) is the most successful algorithm for finding approximate Nash equilibria in imperfect information games.
However, CFR's reliance on full game-tree traversals limits its scalability and generality.
Therefore, the game's state- and action-space is often abstracted (i.e. simplified) for CFR, and the resulting strategy is then mapped back to the full game.
This requires extensive expert-knowledge, is not practical in many games outside of poker, and often converges to highly exploitable policies.
A recently proposed method, Deep CFR, applies deep learning directly to CFR, allowing the agent to intrinsically abstract and generalize over the state-space from samples, without requiring expert knowledge.
In this paper, we introduce Single Deep CFR (SD-CFR), a variant of Deep CFR that has a lower overall approximation error by avoiding the training of an average strategy network.
We show that SD-CFR is more attractive from a theoretical perspective and empirically outperforms Deep CFR with respect to exploitability and one-on-one play in poker.

assistant: text: Better Deep Reinforcement Learning algorithm to approximate Counterfactual Regret Minimization
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Modular Continual Learning in a Unified Visual Environment

Paper content:
A core aspect of human intelligence is the ability to learn new tasks quickly and switch between them flexibly.
Here, we describe a modular continual reinforcement learning paradigm inspired by these abilities.
We first introduce a visual interaction environment that allows many types of tasks to be unified in a single framework.
We then describe a reward map prediction scheme that learns new tasks robustly in the very large state and action spaces required by such an environment.
We investigate how properties of module architecture influence efficiency of task learning, showing that a module motif incorporating specific design principles (e.g. early bottlenecks, low-order polynomial nonlinearities, and symmetry) significantly outperforms more standard neural network motifs, needing fewer training examples and fewer neurons to achieve high levels of performance.
Finally, we present a meta-controller architecture for task switching based on a dynamic neural voting scheme, which allows new modules to use information learned from previously-seen tasks to substantially improve their own learning efficiency.

assistant: text: We propose a neural module approach to continual learning using a unified visual environment with a large action space.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: ROBUST ESTIMATION VIA GENERATIVE ADVERSARIAL NETWORKS

Paper content:
Robust estimation under Huber's $\epsilon$-contamination model has become an important topic in statistics and theoretical computer science.
Rate-optimal procedures such as Tukey's median and other estimators based on statistical depth functions are impractical because of their computational intractability.
In this paper, we establish an intriguing connection between f-GANs and various depth functions through the lens of f-Learning.
Similar to the derivation of f-GAN, we show that these depth functions that lead to rate-optimal robust estimators can all be viewed as variational lower bounds of the total variation distance in the framework of f-Learning.
This connection opens the door of computing robust estimators using tools developed for training GANs.
In particular, we show that a JS-GAN that uses a neural network discriminator with at least one hidden layer is able to achieve the minimax rate of robust mean estimation under Huber's $\epsilon$-contamination model.
Interestingly, the hidden layers of the neural net structure in the discriminator class are shown to be necessary for robust estimation.

assistant: text: GANs are shown to provide us a new effective robust mean estimate against agnostic contaminations with both statistical optimality and practical tractability.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow

Paper content:
Adversarial learning methods have been proposed for a wide range of applications, but the training of adversarial models can be notoriously unstable.
Effectively balancing the performance of the generator and discriminator is critical, since a discriminator that achieves very high accuracy will produce relatively uninformative gradients.
In this work, we propose a simple and general technique to constrain information flow in the discriminator by means of an information bottleneck.
By enforcing a constraint on the mutual information between the observations and the discriminator's internal representation, we can effectively modulate the discriminator's accuracy and maintain useful and informative gradients.
We demonstrate that our proposed variational discriminator bottleneck (VDB) leads to significant improvements across three distinct application areas for adversarial learning algorithms.
Our primary evaluation studies the applicability of the VDB to imitation learning of dynamic continuous control skills, such as running.
We show that our method can learn such skills directly from raw video demonstrations, substantially outperforming prior adversarial imitation learning methods.
The VDB can also be combined with adversarial inverse reinforcement learning to learn parsimonious reward functions that can be transferred and re-optimized in new settings.
Finally, we demonstrate that VDB can train GANs more effectively for image generation, improving upon a number of prior stabilization methods.

assistant: text: Regularizing adversarial learning with an information bottleneck, applied to imitation learning, inverse reinforcement learning, and generative adversarial networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: The Virtual Patch Clamp: Imputing C. elegans Membrane Potentials from Calcium Imaging

Paper content:
We develop a stochastic whole-brain and body simulator of the nematode roundworm Caenorhabditis elegans (C. elegans) and show that it is sufficiently regularizing to allow imputation of latent membrane potentials from partial calcium fluorescence imaging observations.
This is the first attempt we know of to ``complete the circle,'' where an anatomically grounded whole-connectome simulator is used to impute a time-varying ``brain'' state at single-cell fidelity from covariates that are measurable in practice.
Using state of the art Bayesian machine learning methods to condition on readily obtainable data, our method paves the way for neuroscientists to recover interpretable connectome-wide state representations, automatically estimate physiologically relevant parameter values from data, and perform simulations investigating intelligent lifeforms in silico.

assistant: text: We develop a whole-connectome and body simulator for C. elegans and demonstrate joint state-space and parameter inference in the simulator.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Defensive Quantization Layer For Convolutional Network Against Adversarial Attack

Paper content:
Recent research has intensively revealed the vulnerability of deep neural networks, especially for convolutional neural networks (CNNs) on the task of image recognition, through creating adversarial samples which `"slightly" differ from legitimate samples.
This vulnerability indicates that these powerful models are sensitive to specific perturbations and cannot filter out these adversarial perturbations.
In this work, we propose a quantization-based method which enables a CNN to filter out adversarial perturbations effectively.
Notably, different from prior work on input quantization, we apply the quantization in the intermediate layers of a CNN.
Our approach is naturally aligned with the clustering of the coarse-grained semantic information learned by a CNN.
Furthermore, to compensate for the loss of information which is inevitably caused by the quantization, we propose the multi-head quantization, where we project data points to different sub-spaces and perform quantization within each sub-space.
We enclose our design in a quantization layer named as the Q-Layer.
The results obtained on MNIST and Fashion-MNSIT datasets demonstrate that only adding one Q-Layer into a CNN could significantly improve its robustness against both white-box and black-box attacks.

assistant: text: We propose a quantization-based method which regularizes a CNN's learned representations to be automatically aligned with trainable concept matrix hence effectively filtering out adversarial perturbations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Multiple-Attribute Text Rewriting

Paper content:
The dominant approach to unsupervised "style transfer'' in text is based on the idea of learning a latent representation, which is independent of the attributes specifying its "style''.
In this paper, we show that this condition is not necessary and is not always met in practice, even with domain adversarial training that explicitly aims at learning such disentangled representations.
We thus propose a new model that controls several factors of variation in textual data where this condition on disentanglement is replaced with a simpler mechanism based on back-translation.
Our method allows control over multiple attributes, like gender, sentiment, product type, etc., and a more fine-grained control on the trade-off between content preservation and change of style with a pooling operator in the latent space.
Our experiments demonstrate that the fully entangled model produces better generations, even when tested on new and more challenging benchmarks comprising reviews with multiple sentences and multiple attributes.

assistant: text: A system for rewriting text conditioned on multiple controllable attributes
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Approximation and non-parametric estimation of ResNet-type convolutional neural networks via block-sparse fully-connected neural networks

Paper content:
We develop new approximation and statistical learning theories of convolutional neural networks (CNNs) via the ResNet-type structure where the channel size, filter size, and width are fixed.
It is shown that a ResNet-type CNN is a universal approximator and its expression ability is no worse than fully-connected neural networks (FNNs) with a \textit{block-sparse} structure even if the size of each layer in the CNN is fixed.
Our result is general in the sense that we can automatically translate any approximation rate achieved by block-sparse FNNs into that by CNNs.
Thanks to the general theory, it is shown that learning on CNNs satisfies optimality in approximation and estimation of several important function classes.
As applications, we consider two types of function classes to be estimated: the Barron class and H\"older class.
We prove the clipped empirical risk minimization (ERM) estimator can achieve the same rate as FNNs even the channel size, filter size, and width of CNNs are constant with respect to the sample size.
This is minimax optimal (up to logarithmic factors) for the H\"older class.
Our proof is based on sophisticated evaluations of the covering number of CNNs and the non-trivial parameter rescaling technique to control the Lipschitz constant of CNNs to be constructed.

assistant: text: It is shown that ResNet-type CNNs are a universal approximator and its expression ability is not worse than fully connected neural networks (FNNs) with a \textit{block-sparse} structure even if the size of each layer in the CNN is fixed.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: DUAL SPACE LEARNING WITH VARIATIONAL AUTOENCODERS

Paper content:
This paper proposes a dual variational autoencoder (DualVAE), a framework for generating images corresponding to multiclass labels.
Recent research on conditional generative models, such as the Conditional VAE, exhibit image transfer by changing labels.
However, when the dimension of multiclass labels is large, these models cannot change images corresponding to labels, because learning multiple distributions of the corresponding class is necessary to transfer an image.
This leads to the lack of training data.
Therefore, instead of conditioning with labels, we condition with latent vectors that include label information.
DualVAE divides one distribution of the latent space by linear decision boundaries using labels.
Consequently, DualVAE can easily transfer an image by moving a latent vector toward a decision boundary and is robust to the missing values of multiclass labels.
To evaluate our proposed method, we introduce a conditional inception score (CIS) for measuring how much an image changes to the target class.
We evaluate the images transferred by DualVAE using the CIS in CelebA datasets and demonstrate state-of-the-art performance in a multiclass setting.

assistant: text: a new framework using dual space for generating images corresponding to multiclass labels when the number of class is large
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Differentiable Reasoning over a Virtual Knowledge Base

Paper content:
We consider the task of answering complex multi-hop questions using a corpus as a virtual knowledge base (KB).
In particular, we describe a neural module, DrKIT, that traverses textual data like a virtual KB, softly following paths of relations between mentions of entities in the corpus.
At each step the operation uses a combination of sparse-matrix TFIDF indices and maximum inner product search (MIPS) on a special index of contextual representations.
This module is differentiable, so the full system can be trained completely end-to-end using gradient based methods, starting from natural language inputs.
We also describe a pretraining scheme for the index mention encoder by generating hard negative examples using existing knowledge bases.
We show that DrKIT improves accuracy by 9 points on 3-hop questions in the MetaQA dataset, cutting the gap between text-based and KB-based state-of-the-art by 70%.
DrKIT is also very efficient, processing upto 10x more queries per second than existing state-of-the-art QA systems.

assistant: text: Differentiable multi-hop access to a textual knowledge base of indexed contextual representations
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Convolutional Mesh Autoencoders for 3D Face Representation

Paper content:
Convolutional neural networks (CNNs) have achieved state of the art performance on recognizing and representing audio, images, videos and 3D volumes; that is, domains where the input can be characterized by a regular graph structure.
However, generalizing CNNs to irregular domains like 3D meshes is challenging.
Additionally, training data for 3D meshes is often limited.
In this work, we generalize convolutional autoencoders to mesh surfaces.
We perform spectral decomposition of meshes and apply convolutions directly in frequency space.
In addition, we use max pooling and introduce upsampling within the network to represent meshes in a low dimensional space.
We construct a complex dataset of 20,466 high resolution meshes with extreme facial expressions and encode it using our Convolutional Mesh Autoencoder.
Despite limited training data, our method outperforms state-of-the-art PCA models of faces with 50% lower error,  while using 75% fewer parameters.

assistant: text: Convolutional autoencoders generalized to mesh surfaces for encoding and reconstructing extreme 3D facial expressions.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Interpretable Counting for Visual Question Answering

Paper content:
Questions that require counting a variety of objects in images remain a major challenge in visual question answering (VQA).
The most common approaches to VQA involve either classifying answers based on fixed length representations of both the image and question or summing fractional counts estimated from each section of the image.
In contrast, we treat counting as a sequential decision process and force our model to make discrete choices of what to count.
Specifically, the model sequentially selects from detected objects and learns interactions between objects that influence subsequent selections.
A distinction of our approach is its intuitive and interpretable output, as discrete counts are automatically grounded in the image.
Furthermore, our method outperforms the state of the art architecture for VQA on multiple metrics that evaluate counting.

assistant: text: We perform counting for visual question answering; our model produces interpretable outputs by counting directly from detected objects.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Bridging ELBO objective and MMD

Paper content:
One of the challenges in training generative models such as the variational auto encoder (VAE) is avoiding posterior collapse.
When the generator has too much capacity, it is prone to ignoring latent code.
This problem is exacerbated when the dataset is small, and the latent dimension is high.
The root of the problem is the ELBO objective, specifically the Kullback–Leibler (KL) divergence term in objective function.
This paper proposes a new  objective function to replace the KL term with one that emulates the maximum mean discrepancy (MMD) objective.
It also introduces a new technique, named latent clipping, that is used to control distance between samples in latent space.
A probabilistic autoencoder model, named $\mu$-VAE, is designed and trained on MNIST and MNIST Fashion datasets, using the new objective function and is shown to outperform models trained with ELBO and $\beta$-VAE objective.
The $\mu$-VAE is less prone to posterior collapse, and can generate reconstructions and new samples in good quality.
Latent representations learned by $\mu$-VAE are shown to be good and can be used for downstream tasks such as classification.

assistant: text: This paper proposes a new  objective function to replace KL term with one that emulates maximum mean discrepancy (MMD) objective.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Found by NEMO: Unsupervised Object Detection from Negative Examples and Motion

Paper content:
This paper introduces NEMO, an approach to unsupervised object detection that uses motion---instead of image labels---as a cue to learn object detection.
To discriminate between motion of the target object and other changes in the image, it relies on negative examples that show the scene without the object.
The required data can be collected very easily by recording two short videos, a positive one showing the object in motion and a negative one showing the scene without the object.
Without any additional form of pretraining or supervision and despite of occlusions, distractions, camera motion, and adverse lighting, those videos are sufficient to learn object detectors that can be applied to new videos and even generalize to unseen scenes and camera angles.
In a baseline comparison, unsupervised object detection outperforms off-the shelf template matching and tracking approaches that are given an initial bounding box of the object.
The learned object representations are also shown to be accurate enough to capture the relevant information from manipulation task demonstrations, which makes them applicable to learning from demonstration in robotics.
An example of object detection that was learned from 3 minutes of video can be found here: http://y2u.be/u_jyz9_ETz4

assistant: text: Learning to detect objects without image labels from 3 minutes of video
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Distilling the Knowledge of BERT for Text Generation

Paper content:
Large-scale pre-trained language model, such as BERT, has recently achieved great success in a wide range of language understanding tasks.
However, it remains an open question how to utilize BERT for text generation tasks.
In this paper, we present a novel approach to addressing this challenge in a generic sequence-to-sequence (Seq2Seq) setting.
We first propose a new task, Conditional Masked Language Modeling (C-MLM), to enable fine-tuning of BERT on target text-generation dataset.
The fine-tuned BERT (i.e., teacher) is then exploited as extra supervision to improve conventional Seq2Seq models (i.e., student) for text generation.
By leveraging BERT's idiosyncratic bidirectional nature, distilling the knowledge learned from BERT can encourage auto-regressive Seq2Seq models to plan ahead, imposing global sequence-level supervision for coherent text generation.
Experiments show that the proposed approach significantly outperforms strong baselines of Transformer on multiple text generation tasks, including machine translation (MT) and text summarization.
Our proposed model also achieves new state-of-the-art results on the IWSLT German-English and English-Vietnamese MT datasets.

assistant: text: We propose a model-agnostic way to leverage BERT for text generation and achieve improvements over Transformer on 2 tasks over 4 datasets.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On Learning Wire-Length Efficient Neural Networks

Paper content:
Pruning neural networks for wiring length efficiency is considered.
Three techniques are proposed and experimentally tested: distance-based regularization, nested-rank pruning, and layer-by-layer bipartite matching.
The first two algorithms are used in the training and pruning phases, respectively, and the third is used in the arranging neurons phase.
Experiments show that distance-based regularization with weight based pruning tends to perform the best, with or without layer-by-layer bipartite matching.
These results suggest that these techniques may be useful in creating neural networks for implementation in widely deployed specialized circuits.
In this paper we consider the novel problem of learning accurate neural networks that have low total wiring length because this corresponds to energy consumption in the fundamental limit.
We introduce weight-distance regularization, nested rank pruning, and layer-by-layer bipartite matching and show through ablation studies that all of these algorithms are effective, and can even reach state-of-the-art compression ratios.
The results suggests that these techniques may be worth the computational effort if the neural network is to be widely deployed, if significantly lower energy is worth the slight decrease in accuracy, or if the application is to be deployed on either a specialized circuit or general purpose processor.
Table 2 : Average and standard deviation over four trials for Street View House Numbers task on both the wiring length metric (energy) and remaining edges metric (edges).
We note that with the appropriate hyperparameter setting our algorithm outperforms the baseline weight based techniques (p=0) often on both the energy and number of remaining edges metric.
Table 3 : Results of applying the bipartite matching algorithm on the best performing weight-based pruning network and best performing distance-based regularization method before and after applying layer-by-layer bipartite matching.
Average and standard deviation over 4 trials presented.

assistant: text: Three new algorithms with ablation studies to prune neural network to optimize for wiring length, as opposed to number of remaining weights.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: CAYLEYNETS: SPECTRAL GRAPH CNNS WITH COMPLEX RATIONAL FILTERS

Paper content:
The rise of graph-structured data such as social networks, regulatory networks, citation graphs, and functional brain networks, in combination with resounding success of deep learning in various applications, has brought the interest in generalizing deep learning models to non-Euclidean domains.
In this paper, we introduce a new spectral domain convolutional architecture for deep learning on graphs.
The core ingredient of our model is a new class of parametric rational complex functions (Cayley polynomials) allowing to efficiently compute spectral filters on graphs that specialize on frequency bands of interest.
Our model generates rich spectral filters that are localized in space, scales linearly with the size of the input data for sparsely-connected graphs, and can handle different constructions of Laplacian operators.
Extensive experimental results show the superior performance of our approach on spectral image classification, community detection, vertex classification and matrix completion tasks.

assistant: text: A spectral graph convolutional neural network with spectral zoom properties.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A RAD approach to deep mixture models

Paper content:
Flow based models such as Real NVP are an extremely powerful approach to density estimation.
However, existing flow based models are restricted to transforming continuous densities over a continuous input space into similarly continuous distributions over continuous latent variables.
This makes them poorly suited for modeling and representing discrete structures in data distributions, for example class membership or discrete symmetries.
To address this difficulty, we present a normalizing flow architecture which relies on domain partitioning using locally invertible functions, and possesses both real and discrete valued latent variables.
This Real and Discrete (RAD) approach retains the desirable normalizing flow properties of exact sampling, exact inference, and analytically computable probabilities, while at the same time allowing simultaneous modeling of both continuous and discrete structure in a data distribution.

assistant: text: Flow based models, but non-invertible, to also learn discrete variables
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Temporal Difference Models: Model-Free Deep RL for Model-Based Control

Paper content:
Model-free reinforcement learning (RL) has been proven to be a powerful, general tool for learning complex behaviors.
However, its sample efficiency is often impractically large for solving challenging real-world problems, even for off-policy algorithms such as Q-learning.
A limiting factor in classic model-free RL is that the learning signal consists only of scalar rewards, ignoring much of the rich information contained in state transition tuples.
Model-based RL uses this information, by training a predictive model, but often does not achieve the same asymptotic performance as model-free RL due to model bias.
We introduce temporal difference models (TDMs), a family of goal-conditioned value functions that can be trained with model-free learning and used for model-based control.
TDMs combine the benefits of model-free and model-based RL: they leverage the rich information in state transitions to learn very efficiently, while still attaining asymptotic performance that exceeds that of direct model-based RL methods.
Our experimental results show that, on a range of continuous control tasks, TDMs provide a substantial improvement in efficiency compared to state-of-the-art model-based and model-free methods.

assistant: text: We show that a special goal-condition value function trained with model free methods can be used within model-based control, resulting in substantially better sample efficiency and performance.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: LSTOD: Latent Spatial-Temporal Origin-Destination prediction model and its applications in ride-sharing platforms

Paper content:
Origin-Destination (OD) flow data is an important instrument in transportation studies.
Precise prediction of customer demands from each original location to a destination given a series of previous snapshots helps  ride-sharing platforms to better understand their market mechanism.
However, most existing prediction methods ignore the network structure of OD flow data and fail to utilize the topological dependencies among related OD pairs.
In this paper, we propose a latent spatial-temporal origin-destination (LSTOD) model, with a novel convolutional neural network (CNN) filter to learn the spatial features of OD pairs from a graph perspective and an attention structure to capture their long-term periodicity.
Experiments on a real customer request dataset with available OD information from a ride-sharing platform demonstrate the advantage of LSTOD in achieving at least 6.5% improvement in prediction accuracy over the second best model.

assistant: text: We propose a purely convolutional CNN model with attention mechanism to predict spatial-temporal origin-destination flows.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Recurrent Hierarchical Topic-Guided Neural Language Models

Paper content:
To simultaneously capture syntax and semantics from a text corpus, we propose a new larger-context language model that extracts recurrent hierarchical semantic structure via a dynamic deep topic model to guide natural language generation.
Moving beyond a conventional language model that ignores long-range word dependencies and sentence order, the proposed model captures not only intra-sentence word dependencies, but also temporal transitions between sentences and inter-sentence topic dependences.
For inference, we develop a hybrid of stochastic-gradient MCMC and recurrent autoencoding variational Bayes.
Experimental results on a variety of real-world text corpora demonstrate that the proposed model not only outperforms state-of-the-art larger-context language models, but also learns interpretable recurrent multilayer topics and generates diverse sentences and paragraphs that are syntactically correct and semantically coherent.

assistant: text: We introduce a novel larger-context language model to simultaneously captures syntax and semantics, making it capable of generating highly interpretable sentences and paragraphs
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition

Paper content:
As an emerging topic in face recognition, designing margin-based loss functions can increase the feature margin between different classes for enhanced discriminability.
More recently, absorbing the idea of mining-based strategies is adopted to emphasize the misclassified samples and achieve promising results.
However, during the entire training process, the prior methods either do not explicitly emphasize the sample based on its importance that renders the hard samples not fully exploited or explicitly emphasize the effects of semi-hard/hard samples even at the early training stage that may lead to convergence issues.
In this work, we propose a novel Adaptive Curriculum Learning loss (CurricularFace) that embeds the idea of curriculum learning into the loss function to achieve a novel training strategy for deep face recognition, which mainly addresses easy samples in the early training stage and hard ones in the later stage.
Specifically, our CurricularFace adaptively adjusts the relative importance of easy and hard samples during different training stages.
In each stage, different samples are assigned with different importance according to their corresponding difficultness.
Extensive experimental results on popular benchmarks demonstrate the superiority of our CurricularFace over the state-of-the-art competitors.
Code will be available upon publication.

assistant: text: A novel  Adaptive Curriculum Learning loss for deep face recognition
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards an argumentation-based approach to explainable planning

Paper content:
Providing transparency of AI planning systems is crucial for their success in practical applications.
In order to create a transparent system, a user must be able to query it for explanations about its outputs.
We argue that a key underlying principle for this is the use of causality within a planning model, and that argumentation frameworks provide an intuitive representation of such causality.
In this paper, we discuss how argumentation can aid in extracting causalities in plans and models, and how they can create explanations from them.

assistant: text: Argumentation frameworks are used to represent causality of plans/models to be utilized for explanations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: AMRL: Aggregated Memory For Reinforcement Learning

Paper content:
In many partially observable scenarios, Reinforcement Learning (RL) agents must rely on long-term memory in order to learn an optimal policy.
We demonstrate that using techniques from NLP and supervised learning fails at RL tasks due to stochasticity from the environment and from exploration.
Utilizing our insights on the limitations of traditional memory methods in RL, we propose AMRL, a class of models that can learn better policies with greater sample efficiency and are resilient to noisy inputs.
Specifically, our models use a standard memory module to summarize short-term context, and then aggregate all prior states from the standard model without respect to order.
We show that this provides advantages both in terms of gradient decay and signal-to-noise ratio over time.
Evaluating in Minecraft and maze environments that test long-term memory, we find that our model improves average return by 19% over a baseline that has the same number of parameters and by 9% over a stronger baseline that has far more parameters.

assistant: text: In Deep RL, order-invariant functions can be used in conjunction with standard memory modules to improve gradient decay and resilience to noise.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Scalable Laplace Approximation for Neural Networks

Paper content:
We leverage recent insights from second-order optimisation for neural networks to construct a Kronecker factored Laplace approximation to the posterior over the weights of a trained network.
Our approximation requires no modification of the training procedure, enabling practitioners to estimate the uncertainty of their models currently used in production without having to retrain them.
We extensively compare our method to using Dropout and a diagonal Laplace approximation for estimating the uncertainty of a network.
We demonstrate that our Kronecker factored method leads to better uncertainty estimates on out-of-distribution data and is more robust to simple adversarial attacks.
Our approach only requires calculating two square curvature factor matrices for each layer.
Their size is equal to the respective square of the input and output size of the layer, making the method efficient both computationally and in terms of memory usage.
We illustrate its scalability by applying it to a state-of-the-art convolutional network architecture.

assistant: text: We construct a Kronecker factored Laplace approximation for neural networks that leads to an efficient matrix normal distribution over the weights.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: iSparse: Output Informed Sparsification of Neural Networks

Paper content:
Deep neural networks have demonstrated unprecedented success in various knowledge management applications.
However, the networks created are often very complex, with large numbers of trainable edges which require extensive computational resources.
We note that many successful networks nevertheless often contain large numbers of redundant edges.
Moreover, many of these edges may have negligible contributions towards the overall network performance.
In this paper, we propose a novel iSparse framework and experimentally show, that we can sparsify the network, by 30-50%, without impacting the network performance.
iSparse leverages a novel edge significance score, E, to determine the importance of an edge with respect to the final network output.
Furthermore, iSparse can be applied both while training a model or on top of a pre-trained model, making it a  retraining-free approach - leading to a minimal computational overhead.
Comparisons of iSparse against PFEC, NISP, DropConnect, and Retraining-Free on benchmark datasets show that iSparse leads to effective network sparsifications.

assistant: text: iSparse eliminates irrelevant or insignificant network edges with minimal impact on network performance by determining edge importance w.r.t. the final network output.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards Model-Based Contrastive Explanations for Explainable Planning

Paper content:
An important type of question that arises in Explainable Planning is a contrastive question, of the form "Why action A instead of action B?".
These kinds of questions can be answered with a contrastive explanation that compares properties of the original plan containing A against the contrastive plan containing B. An effective explanation of this type serves to highlight the differences between the decisions that have been made by the planner and what the user would expect, as well as to provide further insight into the model and the planning process.
Producing this kind of explanation requires the generation of the contrastive plan.
This paper introduces domain-independent compilations of user questions into constraints.
These constraints are added to the planning model, so that a solution to the new model represents the contrastive plan.
We introduce a formal description of the compilation from user question to constraints in a temporal and numeric PDDL2.1 planning setting.

assistant: text: This paper introduces domain-independent compilations of user questions into constraints for contrastive explanations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: GQ-Net: Training Quantization-Friendly Deep Networks

Paper content:
Network quantization is a model compression and acceleration technique that has become essential to neural network deployment.
Most quantization methods per- form fine-tuning on a pretrained network, but this sometimes results in a large loss in accuracy compared to the original network.
We introduce a new technique to train quantization-friendly networks, which can be directly converted to an accurate quantized network without the need for additional fine-tuning.
Our technique allows quantizing the weights and activations of all network layers down to 4 bits, achieving high efficiency and facilitating deployment in practical settings.
Com- pared to other fully quantized networks operating at 4 bits, we show substantial improvements in accuracy, for example 66.68% top-1 accuracy on ImageNet using ResNet-18, compared to the previous state-of-the-art accuracy of 61.52% Louizos et al. (2019) and a full precision reference accuracy of 69.76%.
We performed a thorough set of experiments to test the efficacy of our method and also conducted ablation studies on different aspects of the method and techniques to improve training stability and accuracy.
Our codebase and trained models are available on GitHub.

assistant: text: We train accurate fully quantized networks using a loss function maximizing full precision model accuracy and minimizing the difference between the full precision and quantized networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On the Adversarial Robustness of Neural Networks without Weight Transport

Paper content:
Neural networks trained with backpropagation, the standard algorithm of deep learning which uses weight transport, are easily fooled by existing gradient-based adversarial attacks.
This class of attacks are based on certain small perturbations of the inputs to make networks misclassify them.
We show that less biologically implausible deep neural networks trained with feedback alignment, which do not use weight transport, can be harder to fool, providing actual robustness.
Tested on MNIST, deep neural networks trained without weight transport (1) have an adversarial accuracy of 98% compared to 0.03% for neural networks trained with backpropagation and (2) generate non-transferable adversarial examples.
However, this gap decreases on CIFAR-10 but is still significant particularly for small perturbation magnitude less than 1 ⁄ 2.

assistant: text: Less biologically implausible deep neural networks trained without weight transport can be harder to fool.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On Federated Learning of Deep Networks from Non-IID Data: Parameter Divergence and the Effects of Hyperparametric Methods

Paper content:
Federated learning, where a global model is trained by iterative parameter averaging of locally-computed updates, is a promising approach for distributed training of deep networks; it provides high communication-efficiency and privacy-preservability, which allows to fit well into decentralized data environments, e.g., mobile-cloud ecosystems.
However, despite the advantages, the federated learning-based methods still have a challenge in dealing with non-IID training data of local devices (i.e., learners).
In this regard, we study the effects of a variety of hyperparametric conditions under the non-IID environments, to answer important concerns in practical implementations:
(i) We first investigate parameter divergence of local updates to explain performance degradation from non-IID data.
The origin of the parameter divergence is also found both empirically and theoretically.
(ii) We then revisit the effects of optimizers, network depth/width, and regularization techniques; our observations show that the well-known advantages of the hyperparameter optimization strategies could rather yield diminishing returns with non-IID data.
(iii) We finally provide the reasons of the failure cases in a categorized way, mainly based on metrics of the parameter divergence.

assistant: text: We investigate the internal reasons of our observations, the diminishing effects of the well-known hyperparameter optimization methods on federated learning from decentralized non-IID data.