ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Robust Learning with Jacobian Regularization

Paper content:
Design of reliable systems must guarantee stability against input perturbations.
In machine learning, such guarantee entails preventing overfitting and ensuring robustness of models against corruption of input data.
In order to maximize stability, we analyze and develop a computationally efficient implementation of Jacobian regularization that increases classification margins of neural networks.
The stabilizing effect of the Jacobian regularizer leads to significant improvements in robustness, as measured against both random and adversarial input perturbations, without severely degrading generalization properties on clean data.

assistant: text: We analyze and develop a computationally efficient implementation of Jacobian regularization that increases the classification margins of neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: R-TRANSFORMER: RECURRENT NEURAL NETWORK ENHANCED TRANSFORMER

Paper content:
Recurrent Neural Networks have long been the dominating choice for sequence modeling.
However, it severely suffers from two issues: impotent in capturing very long-term dependencies and unable to parallelize the sequential computation procedure.
Therefore, many non-recurrent sequence models that are built on convolution and attention operations have been proposed recently.
Notably, models with multi-head attention such as Transformer have demonstrated extreme effectiveness in capturing long-term dependencies in a variety of sequence modeling tasks.
Despite their success, however, these models lack necessary components to model local structures in sequences and heavily rely on position embeddings that have limited effects and require a considerable amount of design efforts.
In this paper, we propose the R-Transformer which enjoys the advantages of both RNNs and the multi-head attention mechanism while avoids their respective drawbacks.
The proposed model can effectively capture both local structures and global long-term dependencies in sequences without any use of position embeddings.
We evaluate R-Transformer through extensive experiments with data from a wide range of domains and the empirical results show that R-Transformer outperforms the state-of-the-art methods by a large margin in most of the tasks.

assistant: text: This paper proposes an effective generic sequence model which leverages the strengths of both RNNs and Multi-head attention.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Evaluating Semantic Representations of Source Code

Paper content:
Learned representations of source code enable various software developer tools, e.g., to detect bugs or to predict program properties.
At the core of code representations often are word embeddings of identifier names in source code, because identifiers account for the majority of source code vocabulary and convey important semantic information.
Unfortunately, there currently is no generally accepted way of evaluating the quality of word embeddings of identifiers, and current evaluations are biased toward specific downstream tasks.
This paper presents IdBench, the first benchmark for evaluating to what extent word embeddings of identifiers represent semantic relatedness and similarity.
The benchmark is based on thousands of ratings gathered by surveying 500 software developers.
We use IdBench to evaluate state-of-the-art embedding techniques proposed for natural language, an embedding technique specifically designed for source code, and lexical string distance functions, as these are often used in current developer tools.
Our results show that the effectiveness of embeddings varies significantly across different embedding techniques and that the best available embeddings successfully represent semantic relatedness.
On the downside, no existing embedding provides a satisfactory representation of semantic similarities, e.g., because embeddings consider identifiers with opposing meanings as similar, which may lead to fatal mistakes in downstream developer tools.
IdBench provides a gold standard to guide the development of novel embeddings that address the current limitations.

assistant: text: A benchmark to evaluate neural embeddings of identifiers in source code.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Multi-Dimensional Explanation of Reviews

Paper content:
Neural models achieved considerable improvement for many natural language processing tasks, but they offer little transparency, and interpretability comes at a cost.
In some domains, automated predictions without justifications have limited applicability.
Recently, progress has been made regarding single-aspect sentiment analysis for reviews, where the ambiguity of a justification is minimal.
In this context, a justification, or mask, consists of (long) word sequences from the input text, which suffice to make the prediction.
Existing models cannot handle more than one aspect in one training and induce binary masks that might be ambiguous.
In our work, we propose a neural model for predicting multi-aspect sentiments for reviews and generates a probabilistic multi-dimensional mask (one per aspect) simultaneously, in an unsupervised and multi-task learning manner.
Our evaluation shows that on three datasets, in the beer and hotel domain, our model outperforms strong baselines and generates masks that are: strong feature predictors, meaningful, and interpretable.

assistant: text: Neural model predicting multi-aspect sentiments and generating a probabilistic multi-dimensional mask simultaneously. Model outperforms strong baselines and generates masks that are: strong feature predictors, meaningful, and interpretable.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Logit Regularization Methods for Adversarial Robustness

Paper content:
While great progress has been made at making neural networks effective across a wide range of tasks, many are surprisingly vulnerable to small, carefully chosen perturbations of their input, known as adversarial examples.
In this paper, we advocate for and experimentally investigate the use of logit regularization techniques as an adversarial defense, which can be used in conjunction with other methods for creating adversarial robustness at little to no cost.
We demonstrate that much of the effectiveness of one recent adversarial defense mechanism can be attributed to logit regularization and show how to improve its defense against both white-box and black-box attacks, in the process creating a stronger black-box attacks against PGD-based models.

assistant: text: Logit regularization methods help explain and improve state of the art adversarial defenses
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: DATNet: Dual Adversarial Transfer for Low-resource Named Entity Recognition

Paper content:
We propose a new architecture termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER).
Specifically, two variants of DATNet, i.e., DATNet-F and DATNet-P, are proposed to explore effective feature fusion between high and low resource.
To address the noisy and imbalanced training data, we propose a novel Generalized Resource-Adversarial Discriminator (GRAD).
Additionally, adversarial training is adopted to boost model generalization.
We examine the effects of different components in DATNet across domains and languages and show that significant improvement can be obtained especially for low-resource data.
Without augmenting any additional hand-crafted features, we achieve new state-of-the-art performances on CoNLL and Twitter NER---88.16% F1 for Spanish, 53.43% F1 for WNUT-2016, and 42.83% F1 for WNUT-2017.

assistant: text: We propose a new  architecture termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER) and achieve new state-of-the-art performances on CoNLL and Twitter NER.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Weightless: Lossy Weight Encoding For Deep Neural Network Compression

Paper content:
The large memory requirements of deep neural networks strain the capabilities of many devices, limiting their deployment and adoption.
Model compression methods effectively reduce the memory requirements of these models, usually through applying transformations such as weight pruning or quantization.
In this paper, we present a novel scheme for lossy weight encoding which complements conventional compression techniques.
The encoding is based on the Bloomier filter, a probabilistic data structure that can save space at the cost of introducing random errors.
Leveraging the ability of neural networks to tolerate these imperfections and by re-training around the errors, the proposed technique, Weightless, can compress DNN weights by up to 496x; with the same model accuracy, this results in up to a 1.51x improvement over the state-of-the-art.

assistant: text: We propose a new way to compress neural networks using probabilistic data structures.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: TRUNCATED HORIZON POLICY SEARCH: COMBINING REINFORCEMENT LEARNING & IMITATION LEARNING

Paper content:
In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle.
We study the effectiveness of the near- optimal cost-to-go oracle on the planning horizon and demonstrate that the cost- to-go oracle shortens the learner’s planning horizon as function of its accuracy: a globally optimal oracle can shorten the planning horizon to one, leading to a one- step greedy Markov Decision Process which is much easier to optimize, while an oracle that is far away from the optimality requires planning over a longer horizon to achieve near-optimal performance.
Hence our new insight bridges the gap and interpolates between imitation learning and reinforcement learning.
Motivated by the above mentioned insights, we propose Truncated HORizon Policy Search (THOR), a method that focuses on searching for policies that maximize the total reshaped reward over a finite planning horizon when the oracle is sub-optimal.
We experimentally demonstrate that a gradient-based implementation of THOR can achieve superior performance compared to RL baselines and IL baselines even when the oracle is sub-optimal.

assistant: text: Combining Imitation Learning and Reinforcement Learning to learn to outperform the expert
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On-Policy Trust Region Policy Optimisation with Replay Buffers

Paper content:
Building upon the recent success of deep reinforcement learning methods, we investigate the possibility of on-policy reinforcement learning improvement by reusing the data from several consecutive policies.
On-policy methods bring many benefits, such as ability to evaluate each resulting policy.
However, they usually discard all the information about the policies which existed before.
In this work, we propose adaptation of the replay buffer concept, borrowed from the off-policy learning setting, to the on-policy algorithms.
To achieve this, the proposed algorithm generalises the Q-, value and advantage functions for data from multiple policies.
The method uses trust region optimisation, while avoiding some of the common problems of the algorithms such as TRPO or ACKTR: it uses hyperparameters to replace the trust region selection heuristics, as well as  the trainable covariance matrix instead of the fixed one.
In many cases, the method not only improves the results comparing to the state-of-the-art trust region on-policy learning algorithms such as ACKTR and TRPO, but also with respect to their off-policy counterpart DDPG.

assistant: text: We investigate the theoretical and practical evidence of on-policy reinforcement learning improvement by reusing the data from several consecutive policies.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Goal-constrained planning domain model formal verification of safety properties

Paper content:
The verification of planning domain models is crucial to ensure the safety, integrity and correctness of planning-based automated systems.
This task is usually performed using model checking techniques.
However, directly applying model checkers to verify planning domain models can result in false positives, i.e. counterexamples that are unreachable by a sound planner when using the domain under verification during a planning task.
In this paper, we discuss the downside of unconstrained planning domain model verification.
We then propose a fail-safe practice for designing planning domain models that can inherently guarantee the safety of the produced plans in case of undetected errors in domain models.
In addition, we demonstrate how model checkers, as well as state trajectory constraints planning techniques, should be used to verify planning domain models so that unreachable counterexamples are not returned.

assistant: text: Why and how to constrain planning domain model verification with planning goals to avoid unreachable counterexamples (false positives verification outcomes).
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Restoration of Video Frames from a Single Blurred Image with Motion Understanding

Paper content:
We propose a novel framework to generate clean video frames from a single motion-blurred image.
While a broad range of literature focuses on recovering a single image from a blurred image, in this work, we tackle a more challenging task i.e.  video restoration from a blurred image.
We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors, and the blurred image as an observation.
Our framework is based on an encoder-decoder structure with spatial transformer network modules to restore a video sequence and its underlying motion in an end-to-end manner.
We design a loss function and regularizers with complementary properties to stabilize the training and analyze variant models of the proposed network.
The effectiveness and transferability of our network are highlighted through a large set of experiments on two different types of datasets: camera rotation blurs generated from panorama scenes and dynamic motion blurs in high speed videos.
Our code and models will be publicly available.

assistant: text: We present a novel unified architecture that restores video frames from a single motion-blurred image in an end-to-end manner.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Weakly Supervised Disentanglement with Guarantees

Paper content:
Learning disentangled representations that correspond to factors of variation in real-world data is critical to interpretable and human-controllable machine learning.
Recently, concerns about the viability of learning disentangled representations in a purely unsupervised manner has spurred a shift toward the incorporation of weak supervision.
However, there is currently no formalism that identifies when and how weak supervision will guarantee disentanglement.
To address this issue, we provide a theoretical framework—including a calculus of disentanglement— to assist in analyzing the disentanglement guarantees (or lack thereof) conferred by weak supervision when coupled with learning algorithms based on distribution matching.
We empirically verify the guarantees and limitations of several weak supervision methods (restricted labeling, match-pairing, and rank-pairing), demonstrating the predictive power and usefulness of our theoretical framework.

assistant: text: We construct a theoretical framework for weakly supervised disentanglement and conducted lots of experiments to back up the theory.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Hardware-aware One-Shot Neural Architecture Search in Coordinate Ascent Framework

Paper content:
Designing accurate and efficient convolutional neural architectures for vast amount of hardware is challenging because hardware designs are complex and diverse.
This paper addresses the hardware diversity challenge in Neural Architecture Search (NAS).
Unlike previous approaches that apply search algorithms on a small, human-designed search space without considering hardware diversity, we propose HURRICANE that explores the automatic hardware-aware search over a much larger search space and a multistep search scheme in coordinate ascent framework, to generate tailored models for different types of hardware.
Extensive experiments on  ImageNet show that our algorithm consistently achieves a much lower inference latency with a similar or better accuracy than state-of-the-art NAS methods on three types of hardware.
Remarkably, HURRICANE achieves a 76.63% top-1 accuracy on ImageNet with a inference latency of only 16.5 ms for DSP, which is a 3.4% higher accuracy and a 6.35x inference speedup than FBNet-iPhoneX.
For VPU, HURRICANE achieves a 0.53% higher top-1 accuracy than Proxyless-mobile with a 1.49x speedup.
Even for well-studied mobile CPU, HURRICANE achieves a 1.63% higher top-1  accuracy than FBNet-iPhoneX with a comparable inference latency.
HURRICANE also reduces the training time by 54.7% on average compared to SinglePath-Oneshot.

assistant: text: We propose HURRICANE to address the challenge of hardware diversity in one-shot neural architecture search
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
Interpreting generative adversarial network (GAN) training as approximate divergence minimization has been theoretically insightful, has spurred discussion, and has lead to theoretically and practically interesting extensions such as f-GANs and Wasserstein GANs.
For both classic GANs and f-GANs, there is an original variant of training and a "non-saturating" variant which uses an alternative form of generator update.
The original variant is theoretically easier to study, but the alternative variant frequently performs better and is recommended for use in practice.
The alternative generator update is often regarded as a simple modification to deal with optimization issues, and it appears to be a common misconception that the two variants minimize the same divergence.
In this short note we derive the divergences approximately minimized by the original and alternative variants of GAN and f-GAN training.
This highlights important differences between the two variants.
For example, we show that the alternative variant of KL-GAN training actually minimizes the reverse KL divergence, and that the alternative variant of conventional GAN training minimizes a "softened" version of the reverse KL.
We hope these results may help to clarify some of the theoretical discussion surrounding the divergence minimization view of GAN training.

assistant: text: Typical GAN training doesn't optimize Jensen-Shannon, but something like a reverse KL divergence.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient Inference on Deep Neural Networks by Dynamic Representations and Decision Gates

Paper content:
The current trade-off between depth and computational cost makes it difficult to adopt deep neural networks for many industrial applications, especially when computing power is limited.
Here, we are inspired by the idea that, while deeper embeddings are needed to discriminate difficult samples, a large number of samples can be well discriminated via much shallower embeddings.
In this study, we introduce the concept of decision gates (d-gate), modules trained to decide whether a sample needs to be projected into a deeper embedding or if an early prediction can be made at the d-gate, thus enabling the computation of dynamic representations at different depths.
The proposed d-gate modules can be integrated with any deep neural network and reduces the average computational cost of the deep neural networks while maintaining modeling accuracy.
Experimental results show that leveraging the proposed d-gate modules led to a ~38% speed-up and ~39% FLOPS reduction on ResNet-101 and ~46% speed-up and $\sim$36\% FLOPS reduction on DenseNet-201 trained on the CIFAR10 dataset with only ~2% drop in accuracy.

assistant: text: This paper introduces a new dynamic feature representation approach to provide a more efficient way to do inference on deep neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Semantic Correspondences from Noisy Data-text Pairs by Local-to-Global Alignments

Paper content:
Learning semantic correspondence between the structured data (e.g., slot-value pairs) and associated texts is a core problem for many downstream NLP applications, e.g., data-to-text generation.
Recent neural generation methods require to use large scale training data.
However, the collected data-text pairs for training are usually loosely corresponded, where texts contain additional or contradicted information compare to its paired input.
In this paper, we propose a local-to-global alignment (L2GA) framework to learn semantic correspondences from loosely related data-text pairs.
First, a local alignment model based on multi-instance learning is applied to build the semantic correspondences within a data-text pair.
Then, a global alignment model built on top of a memory guided conditional random field (CRF) layer is designed to exploit dependencies among alignments in the entire training corpus, where the memory is used to integrate the alignment clues provided by the local alignment model.
Therefore, it is capable of inducing missing alignments for text spans that are not supported by its imperfect paired input.
Experiments on recent restaurant dataset show that our proposed method can improve the alignment accuracy and as a by product, our method is also applicable to induce semantically equivalent training data-text pairs for neural generation models.

assistant: text: We propose a local-to-global alignment framework to learn semantic correspondences from noisy data-text pairs with weak supervision
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Multitask Soft Option Learning

Paper content:
We present Multitask Soft Option Learning (MSOL), a hierarchical multi-task framework based on Planning-as-Inference.
MSOL extends the concept of Options, using separate variational posteriors for each task, regularized by a shared prior.
The learned soft-options are temporally extended, allowing a higher-level master policy to train faster on new tasks by making decisions with lower frequency.
Additionally, MSOL allows fine-tuning of soft-options for new tasks without unlearning previously useful behavior, and avoids problems with local minima in multitask training.
We demonstrate empirically that MSOL significantly outperforms both hierarchical and flat transfer-learning baselines in challenging multi-task environments.

assistant: text: In Hierarchical RL, we introduce the notion of a 'soft', i.e. adaptable, option and show that this helps learning in multitask settings.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Logic and the 2-Simplicial Transformer

Paper content:
We introduce the 2-simplicial Transformer, an extension of the Transformer which includes a form of higher-dimensional attention generalising the dot-product attention, and uses this attention to update entity representations with tensor products of value vectors.
We show that this architecture is a useful inductive bias for logical reasoning in the context of deep reinforcement learning.

assistant: text: We introduce the 2-simplicial Transformer and show that this architecture is a useful inductive bias for logical reasoning in the context of deep reinforcement learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition

Paper content:
The ability to decompose complex multi-object scenes into meaningful abstractions like objects is fundamental to achieve higher-level cognition.
Previous approaches for unsupervised object-oriented scene representation learning are either based on spatial-attention or scene-mixture approaches and limited in scalability which is a main obstacle towards modeling real-world scenes.
In this paper, we propose a generative latent variable model, called SPACE, that provides a uniﬁed probabilistic modeling framework that combines the best of spatial-attention and scene-mixture approaches.
SPACE can explicitly provide factorized object representations for foreground objects while also decomposing background segments of complex morphology.
Previous models are good at either of these, but not both.
SPACE also resolves the scalability problems of previous methods by incorporating parallel spatial-attention and thus is applicable to scenes with a large number of objects without performance degradations.
We show through experiments on Atari and 3D-Rooms that SPACE achieves the above properties consistently in comparison to SPAIR, IODINE, and GENESIS.
Results of our experiments can be found on our project website: https://sites.google.com/view/space-project-page

assistant: text: We propose a generative latent variable model for unsupervised scene decomposition that provides factorized object representation per foreground object while also decomposing background segments of complex morphology.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Boosting Ticket: Towards Practical Pruning for Adversarial Training with Lottery Ticket Hypothesis

Paper content:
Recent research has proposed the lottery ticket hypothesis, suggesting that for a deep neural network, there exist trainable sub-networks performing equally or better than the original model with commensurate training steps.
While this discovery is insightful, finding proper sub-networks requires iterative training and pruning.
The high cost incurred limits the applications of the lottery ticket hypothesis.
We show there exists a subset of the aforementioned sub-networks that converge significantly faster during the training process and thus can mitigate the cost issue.
We conduct extensive experiments to show such sub-networks consistently exist across various model structures for a restrictive setting of hyperparameters (e.g., carefully selected learning rate, pruning ratio, and model capacity).
As a practical application of our findings, we demonstrate that such sub-networks can help in cutting down the total time of adversarial training, a standard approach to improve robustness, by up to 49% on CIFAR-10 to achieve the state-of-the-art robustness.

assistant: text: We show the possibility of pruning to find a small sub-network with significantly higher convergence rate than the full model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Extending the Framework of Equilibrium Propagation to General Dynamics

Paper content:
The biological plausibility of the backpropagation algorithm has long been doubted by neuroscientists.
Two major reasons are that neurons would need to send two different types of signal in the forward and backward phases, and that pairs of neurons would need to communicate through symmetric bidirectional connections.
We present a simple two-phase learning procedure for fixed point recurrent networks that addresses both these issues.
In our model, neurons perform leaky integration and synaptic weights are updated through a local mechanism.
Our learning method extends the framework of Equilibrium Propagation to general dynamics, relaxing the requirement of an energy function.
As a consequence of this generalization, the algorithm does not compute the true gradient of the objective function,
but rather approximates it at a precision which is proven to be directly related to the degree of symmetry of the feedforward and feedback weights.
We show experimentally that the intrinsic properties of the system lead to alignment of the feedforward and feedback weights, and that our algorithm optimizes the objective function.

assistant: text: We describe a biologically plausible learning algorithm for fixed point recurrent networks without tied weights
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Simulating Action Dynamics with Neural Process Networks

Paper content:
Understanding procedural language requires anticipating the causal effects of actions, even when they are not explicitly stated.
In this work, we introduce Neural Process Networks to understand procedural text through (neural) simulation of action dynamics.
Our model complements existing memory architectures with dynamic entity tracking by explicitly modeling actions as state transformers.
The model updates the states of the entities by executing learned action operators.
Empirical results demonstrate that our proposed model can reason about the unstated causal effects of actions, allowing it to provide more accurate contextual information for understanding and generating procedural text, all while offering more interpretable internal representations than existing alternatives.

assistant: text: We propose a new recurrent memory architecture that can track common sense state changes of entities by simulating the causal effects of actions.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Improving the Generalization of Adversarial Training with Domain Adaptation

Paper content:
By injecting adversarial examples into training data, adversarial training is promising for improving the robustness of deep learning models.
However, most existing adversarial training approaches are based on a specific type of adversarial attack.
It may not provide sufficiently representative samples from the adversarial domain, leading to a weak generalization ability on adversarial examples from other attacks.
Moreover, during the adversarial training, adversarial perturbations on inputs are usually crafted by fast single-step adversaries so as to scale to large datasets.
This work is mainly focused on the adversarial training yet efficient FGSM adversary.
In this scenario, it is difficult to train a model with great generalization due to the lack of representative adversarial samples, aka the samples are unable to accurately reflect the adversarial domain.
To alleviate this problem, we propose a novel Adversarial Training with Domain Adaptation (ATDA) method.
Our intuition is to regard the adversarial training on FGSM adversary as a domain adaption task with limited number of target domain samples.
The main idea is to learn a representation that is semantically meaningful and domain invariant on the clean domain as well as the adversarial domain.
Empirical evaluations on Fashion-MNIST, SVHN, CIFAR-10 and CIFAR-100 demonstrate that ATDA can greatly improve the generalization of adversarial training and the smoothness of the learned models, and outperforms state-of-the-art methods on standard benchmark datasets.
To show the transfer ability of our method, we also extend ATDA to the adversarial training on iterative attacks such as PGD-Adversial Training (PAT) and the defense performance is improved considerably.

assistant: text: We propose a novel adversarial training with domain adaptation method that significantly improves the generalization ability on adversarial examples from different attacks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: LEAF: A Benchmark for Federated Settings

Paper content:
Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day.
This wealth of data can help to learn models that can improve the user experience on each device.
However, the scale and heterogeneity of federated data presents new challenges in research areas such as federated learning, meta-learning, and multi-task learning.
As the machine learning community begins to tackle these challenges, we are at a critical time to ensure that developments made in these areas are grounded with realistic benchmarks.
To this end, we propose Leaf, a modular benchmarking framework for learning in federated settings.
Leaf includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all geared towards capturing the obstacles and intricacies of practical federated environments.
With data increasingly being generated on federated networks of remote devices, there is growing interest in empowering on-device applications with models that make use of such data BID25 .
Learning on data generated in federated networks, however, introduces several new obstacles:Statistical: Data is generated on each device in a heterogeneous manner, with each device associated with a different (though perhaps related) underlying data generating distribution.
Moreover, the number of data points typically varies significantly across devices.
We present LEAF, a modular benchmarking framework for learning in federated settings, or ecosystems marked by massively distributed networks of devices.
Learning paradigms applicable in such settings include federated learning, metalearning, multi-task learning, and on-device learning.
LEAF allows researchers and practitioners in these domains to reason about new proposed solutions under more realistic assumptions than previous benchmarks.
We intend to keep LEAF up to date with new datasets, metrics and opensource solutions in order to foster informed and grounded progress in this field TAB1

assistant: text: We present Leaf, a modular benchmarking framework for learning in federated data, with applications to learning paradigms such as federated learning, meta-learning, and multi-task learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Weakly-supervised Knowledge Graph Alignment with Adversarial Learning

Paper content:
Aligning knowledge graphs from different sources or languages, which aims to align both the entity and relation, is critical to a variety of applications such as knowledge graph construction and question answering.
Existing methods of knowledge graph alignment usually rely on a large number of aligned knowledge triplets to train effective models.
However, these aligned triplets may not be available or are expensive to obtain for many domains.
Therefore, in this paper we study how to design fully-unsupervised methods or weakly-supervised methods, i.e., to align knowledge graphs without or with only a few aligned triplets.
We propose an unsupervised framework based on adversarial training, which is able to map the entities and relations in a source knowledge graph to those in a target knowledge graph.
This framework can be further seamlessly integrated with existing supervised methods, where only a limited number of aligned triplets are utilized as guidance.
Experiments on real-world datasets prove the effectiveness of our proposed approach in both the weakly-supervised and unsupervised settings.

assistant: text: This paper studies weakly-supervised knowledge graph alignment with adversarial training frameworks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Robust Conditional Generative Adversarial Networks

Paper content:
Conditional generative adversarial networks (cGAN) have led to large improvements in the task of conditional image generation, which lies at the heart of computer vision.
The major focus so far has been on performance improvement, while there has been little effort in making cGAN more robust to noise.
The regression (of the generator) might lead to arbitrarily large errors in the output, which makes cGAN unreliable for real-world applications.
In this work, we introduce a novel conditional GAN model, called RoCGAN, which leverages structure in the target space of the model to address the issue.
Our model augments the generator with an unsupervised pathway, which promotes the outputs of the generator to span the target manifold even in the presence of intense noise.
We prove that RoCGAN share similar theoretical properties as GAN and experimentally verify that our model outperforms existing state-of-the-art cGAN architectures by a large margin in a variety of domains including images from natural scenes and faces.

assistant: text: We introduce a new type of conditional GAN, which aims to leverage structure in the target space of the generator. We augment the generator with a new, unsupervised pathway to learn the target structure.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Unsupervised Control Through Non-Parametric Discriminative Rewards

Paper content:
Learning to control an environment without hand-crafted rewards or expert data remains challenging and is at the frontier of reinforcement learning research.
We present an unsupervised learning algorithm to train agents to achieve perceptually-specified goals using only a stream of observations and actions.
Our agent simultaneously learns a goal-conditioned policy and a goal achievement reward function that measures how similar a state is to the goal state.
This dual optimization leads to a co-operative game, giving rise to a learned reward function that reflects similarity in controllable aspects of the environment instead of distance in the space of observations.
We demonstrate the efficacy of our agent to learn, in an unsupervised manner, to reach a diverse set of goals on three domains -- Atari, the DeepMind Control Suite and DeepMind Lab.

assistant: text: Unsupervised reinforcement learning method for learning a policy to robustly achieve perceptually specified goals.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Continual Learning via Neural Pruning

Paper content:
Inspired by the modularity and the life-cycle of biological neurons,we introduce Continual Learning via Neural Pruning (CLNP), a new method aimed at lifelong learning in fixed capacity models based on the pruning of neurons of low activity.
In this method, an L1 regulator is used to promote the presence of neurons of zero or low activity whose connections to previously active neurons is permanently severed at the end of training.
Subsequent tasks are trained using these pruned neurons after reinitialization and cause zero deterioration to the performance of previous tasks.
We show empirically that this biologically inspired method leads to state of the art results beating or matching current methods of higher computational complexity.

assistant: text: We use simple and biologically motivated modifications of standard learning techniques to achieve state of the art performance on catastrophic forgetting benchmarks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Skip-Thought GAN: Generating Text through Adversarial Training using Skip-Thought Vectors

Paper content:
In the past few years, various advancements have been made in generative models owing to the formulation of Generative Adversarial Networks (GANs).
GANs have been shown to perform exceedingly well on a wide variety of tasks pertaining to image generation and style transfer.
In the field of Natural Language Processing, word embeddings such as word2vec and GLoVe are state-of-the-art methods for applying neural network models on textual data.
Attempts have been made for utilizing GANs with word embeddings for text generation.
This work presents an approach to text generation using Skip-Thought sentence embeddings in conjunction with GANs based on gradient penalty functions and f-measures.
The results of using sentence embeddings with GANs for generating text conditioned on input information are comparable to the approaches where word embeddings are used.
Numerous efforts have been made in the field of natural language text generation for tasks such as sentiment analysis BID35 and machine translation BID7 BID24 .
Early techniques for generating text conditioned on some input information were template or rule-based engines, or probabilistic models such as n-gram.
In recent times, state-of-the-art results on these tasks have been achieved by recurrent BID23 BID20 and convolutional neural network models trained for likelihood maximization.
This work proposes an Code available at: https://github.com/enigmaeth/skip-thought-gan approach for text generation using Generative Adversarial Networks with Skip-Thought vectors.GANs BID9 are a class of neural networks that explicitly train a generator to produce high-quality samples by pitting against an adversarial discriminative model.
GANs output differentiable values and hence the task of discrete text generation has to use vectors as differentiable inputs.
This is achieved by training the GAN with sentence embedding vectors produced by Skip-Thought , a neural network model for learning fixed length representations of sentences.

assistant: text: Generating text using sentence embeddings from Skip-Thought Vectors with the help of Generative Adversarial Networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Super-AND: A Holistic Approach to Unsupervised Embedding Learning

Paper content:
Unsupervised embedding learning aims to extract good representations from data without the use of human-annotated labels.
Such techniques are apparently in the limelight because of the challenges in collecting massive-scale labels required for supervised learning.
This paper proposes a comprehensive approach, called Super-AND, which is based on the Anchor Neighbourhood Discovery model.
Multiple losses defined in Super-AND make similar samples gather even within a low-density space and keep features invariant against augmentation.
As a result, our model outperforms existing approaches in various benchmark datasets and achieves an accuracy of 89.2% in CIFAR-10 with the Resnet18 backbone network, a 2.9% gain over the state-of-the-art.

assistant: text: We proposed a comprehensive approach for unsupervised embedding learning on the basis of AND algorithm.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adversarial Decomposition of Text Representation

Paper content:
In this paper, we present a method for adversarial decomposition of text representation.
This method can be used to decompose a representation of an input sentence into several independent vectors, where each vector is responsible for a specific aspect of the input sentence.
We evaluate the proposed method on two case studies: the conversion between different social registers and diachronic language change.
We show that the proposed method is capable of fine-grained con- trolled change of these aspects of the input sentence.
For example, our model is capable of learning a continuous (rather than categorical) representation of the style of the sentence, in line with the reality of language use.
The model uses adversarial-motivational training and includes a special motivational loss, which acts opposite to the discriminator and encourages a better decomposition.
Finally, we evaluate the obtained meaning embeddings on a downstream task of para- phrase detection and show that they are significantly better than embeddings of a regular autoencoder.

assistant: text: A method which learns separate representations for the meaning and the form of a sentence
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: MelNet: A Generative Model for Audio in the Frequency Domain

Paper content:
Capturing high-level structure in audio waveforms is challenging because a single second of audio spans tens of thousands of timesteps.
While long-range dependencies are difficult to model directly in the time domain, we show that they can be more tractably modelled in two-dimensional time-frequency representations such as spectrograms.
By leveraging this representational advantage, in conjunction with a highly expressive probabilistic model and a multiscale generation procedure, we design a model capable of generating high-fidelity audio samples which capture structure at timescales which time-domain models have yet to achieve.
We demonstrate that our model captures longer-range dependencies than time-domain models such as WaveNet across a diverse set of unconditional generation tasks, including single-speaker speech generation, multi-speaker speech generation, and music generation.

assistant: text: We introduce an autoregressive generative model for spectrograms and demonstrate applications to speech and music generation
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: LEARNING NEUROSYMBOLIC GENERATIVE MODELS VIA PROGRAM SYNTHESIS

Paper content:
Significant strides have been made toward designing better generative models in recent years.
Despite this progress, however, state-of-the-art approaches are still largely unable to capture complex global structure in data.
For example, images of buildings typically contain spatial patterns such as windows repeating at regular intervals; state-of-the-art generative methods can’t easily reproduce these structures.
We propose to address this problem by incorporating programs representing global structure into the generative model—e.g., a 2D for-loop may represent a configuration of windows.
Furthermore, we propose a framework for learning these models by leveraging program synthesis to generate training data.
On both synthetic and real-world data, we demonstrate that our approach is substantially better than the state-of-the-art at both generating and completing images that contain global structure.

assistant: text: Applying program synthesis to the tasks of image completion and generation within a deep learning framework
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Finite State Representations of Recurrent Policy Networks

Paper content:
Recurrent neural networks (RNNs) are an effective representation of control policies for a wide range of reinforcement and imitation learning problems.
RNN policies, however, are particularly difficult to explain, understand, and analyze due to their use of continuous-valued memory vectors and observation features.
In this paper, we introduce a new technique, Quantized Bottleneck Insertion, to learn finite representations of these vectors and features.
The result is a quantized representation of the RNN that can be analyzed to improve our understanding of memory use and general behavior.
We present results of this approach on synthetic environments and six Atari games.
The resulting finite representations are surprisingly small in some cases, using as few as 3 discrete memory states and 10 observations for a perfect Pong policy.
We also show that these finite policy representations lead to improved interpretability.

assistant: text: Extracting a finite state machine from a recurrent neural network via quantization for the purpose of interpretability with experiments on Atari.