ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: GAN priors for Bayesian inference

Paper content:
Bayesian inference is used extensively to infer and to quantify the uncertainty in a field of interest from a measurement of a related field when the two are linked by a mathematical model.
Despite its many applications, Bayesian inference faces challenges when inferring fields that have discrete representations of large dimension, and/or have prior distributions that are difficult to characterize mathematically.
In this work we demonstrate how the approximate distribution learned by a generative adversarial network (GAN) may be used as a prior in a Bayesian update to address both these challenges.
We demonstrate the efficacy of this approach by inferring and quantifying uncertainty in a physics-based inverse problem and an inverse problem arising in computer vision.
In this latter example, we also demonstrate how the knowledge of the spatial variation of uncertainty may be used to select an optimal strategy of placing the sensors (i.e. taking measurements), where information about the image is revealed one sub-region at a time.

assistant: text: Using GANs as priors for efficient Bayesian inference of complex fields.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
Breathing exercises are an accessible way to manage stress and many mental illness symptoms.
Traditionally, learning breathing exercises involved in-person guidance or audio recordings.
The shift to mobile devices has led to a new way of learning and engaging in breathing exercises as seen in the rise of multiple mobile applications with different breathing representations.
However, limited work has been done to investigate the effectiveness of these visual representations in supporting breathing pace as measured by synchronization.
We utilized a within-subjects study to evaluate four common breathing visuals to understand which is most effective in providing breathing exercise guidance.
Through controlled lab studies and interviews, we identified two representations with clear advantages over the others.
In addition, we found that auditory guidance was not preferred by all users.
We identify potential usability issues with the representations and suggest design guidelines for future development of app-supported breathing training.

assistant: text: We utilized a within-subjects study to evaluate four paced breathing visuals common in mobile apps to understand which is most effective in providing breathing exercise guidance.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: DS-VIC: Unsupervised Discovery of Decision States for Transfer in RL

Paper content:
We learn to identify decision states, namely the parsimonious set of states where decisions meaningfully affect the future states an agent can reach in an environment.
We utilize the VIC framework, which maximizes an agent’s `empowerment’, ie the ability to reliably reach a diverse set of states -- and formulate a sandwich bound on the empowerment objective that allows identification of decision states. Unlike previous work, our decision states are discovered without extrinsic rewards -- simply by interacting with the world.
Our results show that our decision states are:
1) often interpretable, and
2) lead to better exploration on downstream goal-driven tasks in partially observable environments.

assistant: text: Identify decision states (where agent can take actions that matter) without reward supervision, use it for transfer.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Gradient Surgery for Multi-Task Learning

Paper content:
While deep learning and deep reinforcement learning systems have demonstrated impressive results in domains such as image classification, game playing, and robotic control, data efficiency remains a major challenge, particularly as these algorithms learn individual tasks from scratch.
Multi-task learning has emerged as a promising approach for sharing structure across multiple tasks to enable more efficient learning.
However, the multi-task setting presents a number of optimization challenges, making it difficult to realize large efficiency gains compared to learning tasks independently.
The reasons why multi-task learning is so challenging compared to single task learning are not fully understood.
Motivated by the insight that gradient interference causes optimization challenges, we develop a simple and general approach for avoiding interference between gradients from different tasks, by altering the gradients through a technique we refer to as “gradient surgery”.
We propose a form of gradient surgery that projects the gradient of a task onto the normal plane of the gradient of any other task that has a conflicting gradient.
On a series of challenging multi-task supervised and multi-task reinforcement learning problems, we find that this approach leads to substantial gains in efficiency and performance.
Further, it can be effectively combined with previously-proposed multi-task architectures for enhanced performance in a model-agnostic way.

assistant: text: We develop a simple and general approach for avoiding interference between gradients from different tasks, which improves the performance of multi-task learning in both the supervised and reinforcement learning domains.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Variationally Inferred Sampling Through a Refined Bound

Paper content:
A framework for efficient Bayesian inference in probabilistic programs is introduced by embedding a sampler inside a variational posterior approximation.
Its strength lies in both ease of implementation and automatically tuning sampler parameters to speed up mixing time.
Several strategies to approximate the evidence lower bound (ELBO) computation are introduced, including a rewriting of the ELBO objective.
Experimental evidence is shown by performing experiments on an unconditional VAE on density estimation tasks; solving an influence diagram in a high-dimensional space with a conditional variational autoencoder (cVAE) as a deep Bayes classifier; and state-space models for time-series data.
We consider a probabilistic program (PP) to define a distribution p(x, z), where x are observations and z, both latent variables and parameters, and ask queries involving the posterior p(z|x).
This distribution is typically intractable but, conveniently, probabilistic programming languages (PPLs) provide inference engines to approximate it using Monte Carlo methods (e.g. particle Markov Chain Monte Carlo (MCMC) (Andrieu et al., 2010) or Hamiltonian Monte Carlo (HMC) (Neal et al., 2011) ) or variational approximations (e.g. Automatic Differentiation Variational Inference (ADVI) (Kucukelbir et al., 2017) ).
Whereas the latter are biased and underestimate uncertainty, the former may be exceedingly slow depending on the target distribution.
For such reason, over the recent years, there has been an increasing interest in developing more efficient posterior approximations (Nalisnick et al., 2016; Salimans et al., 2015; Tran et al., 2015) .
It is known that the performance of a sampling method depends on the parameters used (Papaspiliopoulos et al., 2007) .
Here we propose a framework to automatically adapt the posterior shape and tune the parameters of a posterior sampler with the aim of boosting Bayesian inference in PPs.
Our framework constitutes a principled way to enhance the flexibility of the variational posterior approximation, yet can be seen also as a procedure to tune the parameters of an MCMC sampler.
Our contributions are a new flexible and unbiased variational approximation to the posterior, which improves an initial variational approximation with a (learnable via automatic differentiation) stochastic process.
Appendix A discusses related work.

assistant: text: We embed SG-MCMC samplers inside a variational approximation
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: DeepProteomics: Protein family classification using Shallow and Deep Networks

Paper content:
The knowledge regarding the function of proteins is necessary as it gives a clear picture of biological processes.
Nevertheless, there are many protein sequences found and added to the databases but lacks functional annotation.
The laboratory experiments take a considerable amount of time for annotation of the sequences.
This arises the need to use computational techniques to classify proteins based on their functions.
In our work, we have collected the data from Swiss-Prot containing 40433 proteins which is grouped into 30 families.
We pass it to recurrent neural network(RNN), long short term memory(LSTM) and gated recurrent unit(GRU) model and compare it by applying trigram with deep neural network and shallow neural network on the same dataset.
Through this approach, we could achieve maximum of around 78% accuracy for the classification of protein families.

assistant: text: Proteins, amino-acid sequences, machine learning, deep learning, recurrent neural network(RNN), long short term memory(LSTM), gated recurrent unit(GRU), deep neural networks
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Batch Normalization is a Cause of Adversarial Vulnerability

Paper content:
Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks.
In many cases it indeed decreases the number of parameter updates required to achieve low training error.
However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard datasets.
Furthermore, substituting weight decay for batch norm is sufficient to nullify the relationship between adversarial vulnerability and the input dimension.
Our work is consistent with a mean-field analysis that found that batch norm causes exploding gradients.

assistant: text: Batch normalization reduces adversarial robustness, as well as general robustness in many cases, particularly to noise corruptions.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Prob2Vec: Mathematical Semantic Embedding for Problem Retrieval in Adaptive Tutoring

Paper content:
We propose a new application of embedding techniques to problem retrieval in adaptive tutoring.
The objective is to retrieve problems similar in mathematical concepts.
There are two challenges: First, like sentences, problems helpful to tutoring are never exactly the same in terms of the underlying concepts.
Instead, good problems mix concepts in innovative ways, while still displaying continuity in their relationships.
Second, it is difficult for humans to determine a similarity score consistent across a large enough training set.
We propose a hierarchical problem embedding algorithm, called Prob2Vec, that consists of an abstraction and an embedding step.
Prob2Vec achieves 96.88\% accuracy on a problem similarity test, in contrast to 75\% from directly applying state-of-the-art sentence embedding methods.
It is surprising that Prob2Vec is able to distinguish very fine-grained differences among problems, an ability humans need time and effort to acquire.
In addition, the sub-problem of concept labeling with imbalanced training data set is interesting in its own right.
It is a multi-label problem suffering from dimensionality explosion, which we propose ways to ameliorate.
We propose the novel negative pre-training algorithm that dramatically reduces false negative and positive ratios for classification, using an imbalanced training data set.

assistant: text: We propose the Prob2Vec method for problem embedding used in a personalized e-learning tool in addition to a data level classification method, called negative pre-training, for cases where the training data set is imbalanced.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards Verified Robustness under Text Deletion Interventions

Paper content:
Neural networks are widely used in Natural Language Processing, yet despite their empirical successes, their behaviour is brittle: they are both over-sensitive to small input changes, and under-sensitive to deletions of large fractions of input text.
This paper aims to tackle under-sensitivity in the context of natural language inference by ensuring that models do not become more confident in their predictions as arbitrary subsets of words from the input text are deleted.
We develop a novel technique for formal verification of this specification for models based on the popular decomposable attention mechanism by employing the efficient yet effective interval bound propagation (IBP) approach.
Using this method we can efficiently prove, given a model, whether a particular sample is free from the under-sensitivity problem.
We compare different training methods to address under-sensitivity, and compare metrics to measure it.
In our experiments on the SNLI and MNLI datasets, we observe that IBP training leads to a significantly improved verified accuracy.
On the SNLI test set, we can verify 18.4% of samples, a substantial improvement over only 2.8% using standard training.

assistant: text: Formal verification of a specification on a model's prediction undersensitivity using Interval Bound Propagation
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Paper content:
Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks.
However, at some point further model increases become harder due to GPU/TPU memory limitations, longer training times, and unexpected model degradation.
To address these problems,  we present two parameter-reduction techniques to lower memory consumption and increase the training speed of BERT.
Comprehensive empirical evidence shows that our proposed methods lead to models that scale much better compared to the original BERT.
We also use a self-supervised loss that focuses on modeling inter-sentence coherence, and show it consistently helps downstream tasks with multi-sentence inputs.
As a result, our best model establishes new state-of-the-art results on the GLUE, RACE, and SQuAD benchmarks while having fewer parameters compared to BERT-large.

assistant: text: A new pretraining method that establishes new state-of-the-art results on the GLUE, RACE, and SQuAD benchmarks while having fewer parameters compared to BERT-large.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning sparse relational transition models

Paper content:
We present a representation for describing transition models in complex uncertain domains using relational rules.
For any action, a rule selects a set of relevant objects and computes a distribution over properties of just those objects in the resulting state given their properties in the previous state.
An iterative greedy algorithm is used to construct a set of deictic references that determine which objects are relevant in any given state.
Feed-forward neural networks are used to learn the transition distribution on the relevant objects' properties.
This strategy is demonstrated to be both more versatile and more sample efficient than learning a monolithic transition model in a simulated domain in which a robot pushes stacks of objects on a cluttered table.

assistant: text: A new approach that learns a representation for describing transition models in complex uncertaindomains using relational rules.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: The Variational Bandwidth Bottleneck: Stochastic Evaluation on an Information Budget

Paper content:
In many applications, it is desirable to extract only the relevant information from complex input data, which involves making a decision about which input features are relevant.
The information bottleneck method formalizes this as an information-theoretic optimization problem by maintaining an optimal tradeoff between compression (throwing away irrelevant input information), and predicting the target.
In many problem settings, including the reinforcement learning problems we consider in this work, we might prefer to compress only part of the input.
This is typically the case when we have a standard conditioning input, such as a state observation, and a ``privileged'' input, which might correspond to the goal of a task, the output of a costly planning algorithm, or communication with another agent.
In such cases, we might prefer to compress the privileged input, either to achieve better generalization (e.g., with respect to goals) or to minimize access to costly information (e.g., in the case of communication).
Practical implementations of the information bottleneck based on variational inference require access to the privileged input in order to compute the bottleneck variable, so although they perform compression, this compression operation itself needs unrestricted, lossless access.
In this work, we propose the variational bandwidth bottleneck, which decides for each example on the estimated value of the privileged information before seeing it, i.e., only based on the standard input, and then accordingly chooses stochastically, whether to access the privileged input or not.
We formulate a tractable approximation to this framework and demonstrate in a series of reinforcement learning experiments that it can improve generalization and reduce access to computationally costly information.

assistant: text: Training agents with adaptive computation based on information bottleneck can promote generalization.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: CEM-RL: Combining evolutionary and gradient-based methods for policy search

Paper content:
Deep neuroevolution and deep reinforcement learning (deep RL) algorithms are two popular approaches to policy search.
The former is widely applicable and rather stable, but suffers from low sample efficiency.
By contrast, the latter is more sample efficient, but the most sample efficient variants are also rather unstable and highly sensitive to hyper-parameter setting.
So far, these families of methods have mostly been compared as competing tools.
However, an emerging approach consists in combining them so as to get the best of both worlds.
Two previously existing combinations use either an ad hoc evolutionary algorithm or a goal exploration process together with the Deep Deterministic Policy Gradient (DDPG) algorithm, a sample efficient off-policy deep RL algorithm.
In this paper, we propose a different combination scheme using the simple cross-entropy
method (CEM) and Twin Delayed Deep Deterministic policy gradient (TD3), another off-policy deep RL algorithm which improves over DDPG.
We evaluate the resulting method, CEM-RL, on a set of benchmarks classically used in deep RL.
We show that CEM-RL benefits from several advantages over its competitors and offers a satisfactory trade-off between performance and sample efficiency.

assistant: text: We propose a new combination of evolution strategy and deep reinforcement learning which takes the best of both worlds
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Composable Planning with Attributes

Paper content:
The tasks that an agent will need to solve often aren’t known during training.
However, if the agent knows which properties of the environment we consider im- portant, then after learning how its actions affect those properties the agent may be able to use this knowledge to solve complex tasks without training specifi- cally for them.
Towards this end, we consider a setup in which an environment is augmented with a set of user defined attributes that parameterize the features of interest.
We propose a model that learns a policy for transitioning between “nearby” sets of attributes, and maintains a graph of possible transitions.
Given a task at test time that can be expressed in terms of a target set of attributes, and a current state, our model infers the attributes of the current state and searches over paths through attribute space to get a high level plan, and then uses its low level policy to execute the plan.
We show in grid-world games and 3D block stacking that our model is able to generalize to longer, more complex tasks at test time even when it only sees short, simple tasks at train time.

assistant: text: Compositional attribute-based planning that generalizes to long test tasks, despite being trained on short & simple tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Elastic-InfoGAN: Unsupervised Disentangled Representation Learning in Imbalanced Data

Paper content:
We propose a novel unsupervised generative model, Elastic-InfoGAN, that learns to disentangle object identity from other low-level aspects in class-imbalanced datasets.
We first investigate the issues surrounding the assumptions about uniformity made by InfoGAN, and demonstrate its ineffectiveness to properly disentangle object identity in imbalanced data.
Our key idea is to make the discovery of the discrete latent factor of variation invariant to identity-preserving transformations in real images, and use that as the signal to learn the latent distribution's parameters.
Experiments on both artificial (MNIST) and real-world (YouTube-Faces) datasets demonstrate the effectiveness of our approach in imbalanced data by:
(i) better disentanglement of object identity as a latent factor of variation; and
(ii) better approximation of class imbalance in the data, as reflected in the learned parameters of the latent distribution.

assistant: text: Elastic-InfoGAN is a modification of InfoGAN that learns, without any supervision, disentangled representations in class imbalanced data
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Explaining AlphaGo: Interpreting Contextual Effects in Neural Networks

Paper content:
This paper presents two methods to disentangle and interpret contextual effects that are encoded in a pre-trained deep neural network.
Unlike convolutional studies that visualize image appearances corresponding to the network output or a neural activation from a global perspective, our research aims to clarify how a certain input unit (dimension) collaborates with other units (dimensions) to constitute inference patterns of the neural network and thus contribute to the network output.
The analysis of local contextual effects w.r.t. certain input units is of special values in real applications.
In particular, we used our methods to explain the gaming strategy of the alphaGo Zero model in experiments, and our method successfully disentangled the rationale of each move during the game.

assistant: text: This paper presents methods to disentangle and interpret contextual effects that are encoded in a deep neural network.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep Amortized Variational Inference for Multivariate Time Series Imputation with Latent Gaussian Process Models

Paper content:
Multivariate time series with missing values are common in areas such as healthcare and finance, and have grown in number and complexity over the years.
This raises the question whether deep learning methodologies can outperform classical data imputation methods in this domain.
However, naive applications of deep learning fall short in giving reliable confidence estimates and lack interpretability.
We propose a new deep sequential latent variable model for dimensionality reduction and data imputation.
Our modeling assumption is simple and interpretable: the high dimensional time series has a lower-dimensional representation which evolves smoothly in time according to a Gaussian process.
The non-linear dimensionality reduction in the presence of missing data is achieved using a VAE approach with a novel structured variational approximation.
We demonstrate that our approach outperforms several classical and deep learning-based data imputation methods on high-dimensional data from the domains of computer vision and healthcare, while additionally improving the smoothness of the imputations and providing interpretable uncertainty estimates.

assistant: text: We perform amortized variational inference on a latent Gaussian process model to achieve superior imputation performance on multivariate time series with missing data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Automatic Labeling of Data for Transfer Learning

Paper content:
Transfer learning uses trained weights from a source model as the initial weightsfor the training of a target dataset.
A well chosen source with a large numberof labeled data leads to significant improvement in accuracy.
We demonstrate atechnique that automatically labels large unlabeled datasets so that they can trainsource models for transfer learning.
We experimentally evaluate this method, usinga baseline dataset of human-annotated ImageNet1K labels, against five variationsof this technique.
We show that the performance of these automatically trainedmodels come within 17% of baseline on average.
In many domains, the task performance of deep learning techniques is heavily dependent on the number of labeled examples, which are difficult and expensive to acquire.
This demand for large labeled datasets has inspired alternative techniques, such as weak supervision or automated labeling, whose algorithms create plausible labels to be used to guide supervised training on other tasks.In this work, we develop a content-aware model-selection technique for transfer learning.
We take an unlabeled data point (here, an unlabeled image), and compute its distance to the average response of a number of specialized deep learning models, such as those trained for "animal", "person", or "sport".
We then create a "pseudolabel" for the point, consisting of a short ordered sequence of the most appropriate model names, like "animal-plant-building".
We use these synthetic labels to augment the ground truth labels.
We validate the technique by applying it to the ImageNet1K dataset, as well as on a number of other large, unlabeled datasets.
We have shown that generation of content-aware pseudolabels can provide transfer performance approaching that of human labels, and that models trained on psuedolabels can be used as source models for transfer learning.
The automated approach presented here suggests that the internal representations of content models trained on specialized datasets contain some descriptive features of those datasets.
By treating each of these specialized representations as a "word" in a longer "sentence" that describes a category of images, we can create labels such as a "music-weapon-person" to describe a suit of armor, or a "tree-animal-fungus" to describe an elephant.
These rich labels capture features of these objects such as visual information about the materials they are made out of, that better describe the contents than reliance on a single label would produce.
Using multiple, content-aware models to achieve greater descriptive power may be a valuable future avenue of research.

assistant: text: A technique for automatically labeling large unlabeled datasets so that they can train source models for transfer learning and its experimental evaluation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Certifying Some Distributional Robustness with Principled Adversarial Training

Paper content:
Neural networks are vulnerable to adversarial examples and researchers have proposed many heuristic attack and defense mechanisms.
We address this problem through the principled lens of distributionally robust optimization, which guarantees performance under adversarial input perturbations.
By considering a Lagrangian penalty formulation of perturbing the underlying data distribution in a Wasserstein ball, we provide a training procedure that augments model parameter updates with worst-case perturbations of training data.
For smooth losses, our procedure provably achieves moderate levels of robustness with little computational or statistical cost relative to empirical risk minimization.
Furthermore, our statistical guarantees allow us to efficiently certify robustness for the population loss.
For imperceptible perturbations, our method matches or outperforms heuristic approaches.

assistant: text: We provide a fast, principled adversarial training procedure with computational and statistical performance guarantees.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
While machine learning models achieve human-comparable performance on sequential data, exploiting structured knowledge is still a challenging problem.
Spatio-temporal graphs have been proved to be a useful tool to abstract interaction graphs and previous works exploits carefully designed feed-forward architecture to preserve such structure.
We argue to scale such network design to real-world problem, a model needs to automatically learn a meaningful representation of the possible relations.
Learning such interaction structure is not trivial: on the one hand, a model has to discover the hidden relations between different problem factors in an unsupervised way; on the other hand, the mined relations have to be interpretable.
In this paper, we propose an attention module able to project a graph sub-structure in a fixed size embedding, preserving the influence that the neighbours exert on a given vertex.
On a comprehensive evaluation done on real-world as well as toy task, we found our model competitive against strong baselines.

assistant: text: A graph neural network able to automatically learn and leverage a dynamic interactive graph structure
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Training wide residual networks for deployment using a single bit for each weight

Paper content:
For fast and energy-efficient deployment of trained deep neural networks on resource-constrained embedded hardware,  each learned weight parameter should ideally be represented and stored using a single bit.
Error-rates usually increase when this requirement is imposed.
Here, we report large improvements in error rates on multiple datasets, for deep convolutional neural networks deployed with 1-bit-per-weight.
Using wide residual networks as our main baseline, our approach simplifies existing methods that binarize weights by applying the sign function in training; we apply  scaling factors for each layer with constant unlearned values equal to the layer-specific standard deviations used for initialization.
For CIFAR-10, CIFAR-100 and ImageNet, and models with 1-bit-per-weight requiring less than 10 MB of parameter memory, we achieve error rates of 3.9%, 18.5% and 26.0% / 8.5% (Top-1 / Top-5) respectively.
We also considered MNIST, SVHN and ImageNet32, achieving 1-bit-per-weight test results of 0.27%, 1.9%, and 41.3% / 19.1%  respectively.
For CIFAR, our error rates halve previously reported values, and are within about 1% of our error-rates for the same network with full-precision weights.
For networks that overfit, we also show significant improvements in error rate by not learning batch normalization scale and offset parameters.
This applies to both full precision and 1-bit-per-weight networks.
Using a warm-restart learning-rate schedule, we found that training for 1-bit-per-weight is just as fast as full-precision networks, with better accuracy than standard schedules, and achieved about 98%-99% of peak performance in just 62 training epochs for CIFAR-10/100.
For full training code and trained models in MATLAB, Keras and PyTorch see https://github.com/McDonnell-Lab/1-bit-per-weight/ .

assistant: text: We train wide residual networks that can be immediately deployed using only a single bit for each convolutional weight, with signficantly better accuracy than past methods.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: CrossLang: the system of cross-lingual plagiarism detection

Paper content:
Plagiarism and text reuse become more available with the Internet development.
Therefore it is important to check scientific papers for the fact of cheating, especially in Academia.
Existing systems of plagiarism detection show the good performance and have a huge source databases.
Thus now it is not enough just to copy the text as is from the source document to get the original work.
Therefore, another type of plagiarism become popular -- cross-lingual plagiarism.
We present a CrossLang system for such kind of plagiarism detection for English-Russian language pair.

assistant: text: A system for cross-lingual (English-Russian) plagiarism detection
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Progressive Compressed Records: Taking a Byte Out of Deep Learning Data

Paper content:
Deep learning training accesses vast amounts of data at high velocity, posing challenges for datasets retrieved over commodity networks and storage devices.
We introduce a way to dynamically reduce the overhead of fetching and transporting training data with a method we term Progressive Compressed Records (PCRs).
PCRs deviate from previous formats by leveraging progressive compression to split each training example into multiple examples of increasingly higher fidelity, without adding to the total data size.
Training examples of similar fidelity are grouped together, which reduces both the system overhead and data bandwidth needed to train a model.
We show that models can be trained on aggressively compressed representations of the training data and still retain high accuracy, and that PCRs can enable a 2x speedup on average over baseline formats using JPEG compression.
Our results hold across deep learning architectures for a wide range of datasets: ImageNet, HAM10000, Stanford Cars, and CelebA-HQ.

assistant: text: We propose a simple, general, and space-efficient data format to accelerate deep learning training by allowing sample fidelity to be dynamically selected at training time
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Parallel Recurrent Data Augmentation for GAN training with Limited and Diverse Data

Paper content:
The need for large amounts of training image data with clearly defined features is a major obstacle to applying generative adversarial networks(GAN) on image generation where training data is limited but diverse, since insufficient latent feature representation in the already scarce data often leads to instability and mode collapse during GAN training.
To overcome the hurdle of limited data when applying GAN to limited datasets, we propose in this paper the strategy of \textit{parallel recurrent data augmentation}, where the GAN model progressively enriches its training set with sample images constructed from GANs trained in parallel at consecutive training epochs.
Experiments on a variety of small yet diverse datasets demonstrate that our method, with little model-specific considerations, produces images of better quality as compared to the images generated  without such strategy.
The source code and generated images of this paper will be made public after review.

assistant: text: We introduced a novel, simple, and efficient data augmentation method that boosts the performances of existing GANs when training data is limited and diverse.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: The International Competition on Knowledge Engineering for Planning and Scheduling: Food for Thoughts (and Call to Action)

Paper content:
The International Competition on Knowledge Engineering for Planning and Scheduling (ICKEPS) plays a pivotal role in fostering the development of new Knowledge Engineering (KE) tools, and in emphasising the importance of principled approaches for all the different KE aspects that are needed for the successful long-term use of planning in real-world applications.
In this paper, as an exercise in synthesis and for the sake of stimulating thoughts and discussion, we review the format of previous ICKEPS, to suggest alternative formats for future competitions, ideally to motivate someone to step up and organise the next ones.

assistant: text: Ideas for future ICKEPS
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Neuron Hierarchical Networks

Paper content:
In this paper, we propose a neural network framework called neuron hierarchical network (NHN), that evolves beyond the hierarchy in layers, and concentrates on the hierarchy of neurons.
We observe mass redundancy in the weights of both handcrafted and randomly searched architectures.
Inspired by the development of human brains, we prune low-sensitivity neurons in the model and add new neurons to the graph, and the relation between individual neurons are emphasized and the existence of layers weakened.
We propose a process to discover the best base model by random architecture search, and discover the best locations and connections of the added neurons by evolutionary search.
Experiment results show that the NHN achieves higher test accuracy on Cifar-10 than state-of-the-art handcrafted and randomly searched architectures, while requiring much fewer parameters and less searching time.

assistant: text: By breaking the layer hierarchy, we propose a 3-step approach to the construction of neuron-hierarchy networks that outperform NAS, SMASH and hierarchical representation with fewer parameters and shorter searching time.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Gradient Descent Maximizes the Margin of Homogeneous Neural Networks

Paper content:
In this paper, we study the implicit regularization of the gradient descent algorithm in homogeneous neural networks, including fully-connected and convolutional neural networks with ReLU or LeakyReLU activations.
In particular, we study the gradient descent or gradient flow (i.e., gradient descent with infinitesimal step size) optimizing the logistic loss or cross-entropy loss of any homogeneous model (possibly non-smooth), and show that if the training loss decreases below a certain threshold, then we can define a smoothed version of the normalized margin which increases over time.
We also formulate a natural constrained optimization problem related to margin maximization, and prove that both the normalized margin and its smoothed version converge to the objective value at a KKT point of the optimization problem.
Our results generalize the previous results for logistic regression with one-layer or multi-layer linear networks, and provide more quantitative convergence results with weaker assumptions than previous results for homogeneous smooth neural networks.
We conduct several experiments to justify our theoretical finding on MNIST and CIFAR-10 datasets.
Finally, as margin is closely related to robustness, we discuss potential benefits of training longer for improving the robustness of the model.

assistant: text: We study the implicit bias of gradient descent and prove under a minimal set of assumptions that the parameter direction of homogeneous models converges to KKT points of a natural margin maximization problem.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Manifold Mixup: Learning Better Representations by Interpolating Hidden States

Paper content:
Deep networks often perform well on the data distribution on which they are trained, yet give incorrect (and often very confident) answers when evaluated on points from off of the training distribution.
This is exemplified by the adversarial examples phenomenon but can also be seen in terms of model generalization and domain shift.
Ideally, a model would assign lower confidence to points unlike those from the training distribution.
We propose a regularizer which addresses this issue by training with interpolated hidden states and encouraging the classifier to be less confident at these points.
Because the hidden states are learned, this has an important effect of encouraging the hidden states for a class to be concentrated in such a way so that interpolations within the same class or between two different classes do not intersect with the real data points from other classes.
This has a major advantage in that it avoids the underfitting which can result from interpolating in the input space.
We prove that the exact condition for this problem of underfitting to be avoided by Manifold Mixup is that the dimensionality of the hidden states exceeds the number of classes, which is often the case in practice.
Additionally, this concentration can be seen as making the features in earlier layers more discriminative.
We show that despite requiring no significant additional computation, Manifold Mixup achieves large improvements over strong baselines in supervised learning, robustness to single-step adversarial attacks, semi-supervised learning, and Negative Log-Likelihood on held out samples.

assistant: text: A method for learning better representations, that acts as a regularizer and despite its no significant additional computation cost , achieves improvements over strong baselines on Supervised and Semi-supervised Learning tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: BAM! Born-Again Multi-Task Networks for Natural Language Understanding

Paper content:
It can be challenging to train multi-task neural networks that outperform or even match their single-task counterparts.
To help address this, we propose using knowledge distillation where single-task models teach a multi-task model.
We enhance this training with teacher annealing, a novel method that gradually transitions the model from distillation to supervised learning, helping the multi-task model surpass its single-task teachers.
We evaluate our approach by multi-task fine-tuning BERT on the GLUE benchmark.
Our method consistently improves over standard single-task and multi-task training.

assistant: text: distilling single-task models into a multi-task model improves natural language understanding performance
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Autoencoder-based Initialization for Recurrent Neural Networks with a Linear Memory

Paper content:
Orthogonal recurrent neural networks address the vanishing gradient problem by parameterizing the recurrent connections using an orthogonal matrix.
This class of models is particularly effective to solve tasks that require the memorization of long sequences.
We propose an alternative solution based on explicit memorization using linear autoencoders for sequences.
We show how a recently proposed recurrent architecture, the Linear Memory Network, composed of a nonlinear feedforward layer and a separate linear recurrence, can be used to solve hard memorization tasks.
We propose an initialization schema that sets the weights of a recurrent architecture to approximate a linear autoencoder of the input sequences, which can be found with a closed-form solution.
The initialization schema can be easily adapted to any recurrent architecture.
We argue that this approach is superior to a random orthogonal initialization due to the autoencoder, which allows the memorization of long sequences even before training.
The empirical analysis show that our approach achieves competitive results against alternative orthogonal models, and the LSTM, on sequential MNIST, permuted MNIST and TIMIT.

assistant: text: We show how to initialize recurrent architectures with the closed-form solution of a linear autoencoder for sequences. We show the advantages of this approach compared to orthogonal RNNs.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adversarial Imitation via Variational Inverse Reinforcement Learning

Paper content:
We consider a problem of learning the reward and policy from expert examples under unknown dynamics.
Our proposed method builds on the framework of generative adversarial networks and introduces the empowerment-regularized maximum-entropy inverse reinforcement learning to learn near-optimal rewards and policies.
Empowerment-based regularization prevents the policy from overfitting to expert demonstrations, which advantageously leads to more generalized behaviors that result in learning near-optimal rewards.
Our method simultaneously learns empowerment through variational information maximization along with the reward and policy under the adversarial learning formulation.
We evaluate our approach on various high-dimensional complex control tasks.
We also test our learned rewards in challenging transfer learning problems where training and testing environments are made to be different from each other in terms of dynamics or structure.
The results show that our proposed method not only learns near-optimal rewards and policies that are matching expert behavior but also performs significantly better than state-of-the-art inverse reinforcement learning algorithms.

assistant: text: Our method introduces the empowerment-regularized maximum-entropy inverse reinforcement learning to learn near-optimal rewards and policies from expert demonstrations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: The Laplacian in RL: Learning Representations with Efficient Approximations

Paper content:
The smallest eigenvectors of the graph Laplacian are well-known to provide a succinct representation of the geometry of a weighted graph.
In reinforcement learning (RL), where the weighted graph may be interpreted as the state transition process induced by a behavior policy acting on the environment, approximating the eigenvectors of the Laplacian provides a promising approach to state representation learning.
However, existing methods for performing this approximation are ill-suited in general RL settings for two main reasons:  First, they are computationally expensive, often requiring operations on large matrices.
Second, these methods lack adequate justification beyond simple, tabular, finite-state settings.
In this paper, we present a fully general and scalable method for approximating the eigenvectors of the Laplacian in a model-free RL context.
We systematically evaluate our approach and empirically show that it generalizes beyond the tabular, finite-state setting.
Even in tabular, finite-state settings, its ability to approximate the eigenvectors outperforms previous proposals.
Finally, we show the potential benefits of using a Laplacian representation learned using our method in goal-achieving RL tasks, providing evidence that our technique can be used to significantly improve the performance of an RL agent.

assistant: text: We propose a scalable method to approximate the eigenvectors of the Laplacian in the reinforcement learning context and we show that the learned representations can improve the performance of an RL agent.