ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Continuous Meta-Learning without Tasks

Paper content:
Meta-learning is a promising strategy for learning to efficiently learn within new tasks, using data gathered from a distribution of tasks.
However, the meta-learning literature thus far has focused on the task segmented setting, where at train-time, offline data is assumed to be split according to the underlying task, and at test-time, the algorithms are optimized to learn in a single task.
In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task.
We present meta-learning via online changepoint analysis (MOCA), an approach which augments a meta-learning algorithm with a differentiable Bayesian changepoint detection scheme.
The framework allows both training and testing directly on time series data without segmenting it into discrete tasks.
We demonstrate the utility of this approach on a nonlinear meta-regression benchmark as well as two meta-image-classification benchmarks.

assistant: text: The paper considers the meta-learning in the task un-segmented setting and apply Bayesian online change point detection with meta-learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Stochastic Prediction of Multi-Agent Interactions from Partial Observations

Paper content:
We present a method which learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of interacting agents.
Our method is based on a graph-structured variational recurrent neural network, which is trained end-to-end to infer the current state of the (partially observed) world, as well as to forecast future states.
We show that our method outperforms various baselines on two sports datasets, one based on real basketball trajectories, and one generated by a soccer game engine.

assistant: text: We present a method which learns to integrate temporal information and ambiguous visual information in the context of interacting agents.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Few-Shot Regression via Learned Basis Functions

Paper content:
The recent rise in popularity of few-shot learning algorithms has enabled models to quickly adapt to new tasks based on only a few training samples.
Previous few-shot learning works have mainly focused on classification and reinforcement learning.
In this paper, we propose a few-shot meta-learning system that focuses exclusively on regression tasks.
Our model is based on the idea that the degree of freedom of the unknown function can be significantly reduced if it is represented as a linear combination of a set of appropriate basis functions.
This enables a few labelled samples to approximate the function.
We design a Feature Extractor network to encode basis functions for a task distribution, and a  Weights Generator to generate the weight vector for a novel task.
We show that our model outperforms the current state of the art meta-learning methods in various regression tasks.

assistant: text: We propose a few-shot learning model that is tailored specifically for regression tasks
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep Graph Matching Consensus

Paper content:
This work presents a two-stage neural architecture for learning and refining structural correspondences between graphs.
First, we use localized node embeddings computed by a graph neural network to obtain an initial ranking of soft correspondences between nodes.
Secondly, we employ synchronous message passing networks to iteratively re-rank the soft correspondences to reach a matching consensus in local neighborhoods between graphs.
We show, theoretically and empirically, that our message passing scheme computes a well-founded measure of consensus for corresponding neighborhoods, which is then used to guide the iterative re-ranking process.
Our purely local and sparsity-aware architecture scales well to large, real-world inputs while still being able to recover global correspondences consistently.
We demonstrate the practical effectiveness of our method on real-world tasks from the fields of computer vision and entity alignment between knowledge graphs, on which we improve upon the current state-of-the-art.

assistant: text: A two-stage GNN-based architecture to establish correspondences between two graphs that performs well on real-world tasks of image matching and knowledge graph entity alignment.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning to Imagine Manipulation Goals for Robot Task Planning

Paper content:
Prospection is an important part of how humans come up with new task plans, but has not been explored in depth in robotics.
Predicting multiple task-level is a challenging problem that involves capturing both task semantics and continuous variability over the state of the world.
Ideally, we would combine the ability of machine learning to leverage big data for learning the semantics of a task, while using techniques from task planning to reliably generalize to new environment.
In this work, we propose a method for learning a model encoding just such a representation for task planning.
We learn a neural net that encodes the k most likely outcomes from high level actions from a given world.
Our approach creates comprehensible task plans that allow us to predict changes to the environment many time steps into the future.
We demonstrate this approach via application to a stacking task in a cluttered environment, where the robot must select between different colored blocks while avoiding obstacles, in order to perform a task.
We also show results on a simple navigation task.
Our algorithm generates realistic image and pose predictions at multiple points in a given task.

assistant: text: Evaluates the quality of a proposed generative predictive model to generate plans for robot execution.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Local Stability and Performance of Simple Gradient Penalty $\mu$-Wasserstein GAN

Paper content:
Wasserstein GAN(WGAN) is a model that minimizes the Wasserstein distance between a data distribution and sample distribution.
Recent studies have proposed stabilizing the training process for the WGAN and implementing the Lipschitz constraint.
In this study, we prove the local stability of optimizing the simple gradient penalty $\mu$-WGAN(SGP $\mu$-WGAN) under suitable assumptions regarding the equilibrium and penalty measure $\mu$.
The measure valued differentiation concept is employed to deal with the derivative of the penalty terms, which is helpful for handling abstract singular measures with lower dimensional support.
Based on this analysis, we claim that penalizing the data manifold or sample manifold is the key to regularizing the original WGAN with a gradient penalty.
Experimental results obtained with unintuitive penalty measures that satisfy our assumptions are also provided to support our theoretical results.

assistant: text: This paper deals with stability of simple gradient penalty $\mu$-WGAN optimization by introducing a concept of measure valued differentiation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Prestopping: How Does Early Stopping Help Generalization Against Label Noise?

Paper content:
Noisy labels are very common in real-world training data, which lead to poor generalization on test data because of overfitting to the noisy labels.
In this paper, we claim that such overfitting can be avoided by "early stopping" training a deep neural network before the noisy labels are severely memorized.
Then, we resume training the early stopped network using a "maximal safe set," which maintains a collection of almost certainly true-labeled samples at each epoch since the early stop point.
Putting them all together, our novel two-phase training method, called Prestopping, realizes noise-free training under any type of label noise for practical use.
Extensive experiments using four image benchmark data sets verify that our method significantly outperforms four state-of-the-art methods in test error by 0.4–8.2 percent points under existence of real-world noise.

assistant: text: We propose a novel two-phase training approach based on "early stopping" for robust training on noisy labels.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: High Fidelity Speech Synthesis with Adversarial Networks

Paper content:
Generative adversarial networks have seen rapid development in recent years and have led to remarkable improvements in generative modelling of images.
However, their application in the audio domain has received limited attention,
and autoregressive models, such as WaveNet, remain the state of the art in generative modelling of audio signals such as human speech.
To address this paucity, we introduce GAN-TTS, a Generative Adversarial Network for Text-to-Speech.
Our architecture is composed of a conditional feed-forward generator producing raw speech audio, and an ensemble of discriminators which operate on random windows of different sizes.
The discriminators analyse the audio both in terms of general realism, as well as how well the audio corresponds to the utterance that should be pronounced.
To measure the performance of GAN-TTS, we employ both subjective human evaluation (MOS - Mean Opinion Score), as well as novel quantitative metrics (Fréchet DeepSpeech Distance and Kernel DeepSpeech Distance), which we find to be well correlated with MOS.
We show that GAN-TTS is capable of generating high-fidelity speech with naturalness comparable to the state-of-the-art models, and unlike autoregressive models, it is highly parallelisable thanks to an efficient feed-forward generator.
Listen to GAN-TTS reading this abstract at http://tiny.cc/gantts.

assistant: text: Solves the GAN challenge in raw waveform synthesis and begins to close the existing performance gap between autoregressive models and GANs for raw audios.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Scaling characteristics of sequential multitask learning: Networks naturally learn to learn

Paper content:
We explore the behavior of a standard convolutional neural net in a setting that introduces classification tasks sequentially and requires the net to master new tasks while preserving mastery of previously learned tasks.
This setting corresponds to that which human learners face as they acquire domain expertise, for example, as an individual reads a textbook chapter-by-chapter.
Through simulations involving sequences of 10 related tasks, we find reason for optimism that nets will scale well as they advance from having a single skill to becoming domain experts.
We observed two key phenomena.
First, forward facilitation---the accelerated learning of task n+1 having learned n previous tasks---grows with n. Second, backward interference---the forgetting of the n previous tasks when learning task n+1---diminishes with n.  Forward facilitation is the goal of research on metalearning, and reduced backward interference is the goal of research on ameliorating catastrophic forgetting.
We find that both of these goals are attained simply through broader exposure to a domain.

assistant: text: We study the behavior of a CNN as it masters new tasks while preserving mastery for previously learned tasks
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Q-learning with UCB Exploration is Sample Efficient for Infinite-Horizon MDP

Paper content:
A fundamental question in reinforcement learning is whether model-free algorithms are sample efficient.
Recently,  Jin et al. (2018) proposed a Q-learning algorithm with UCB exploration policy, and proved it has nearly optimal regret bound for finite-horizon episodic MDP.
In this paper, we adapt Q-learning with UCB-exploration bonus to infinite-horizon MDP with discounted rewards \emph{without} accessing a generative model.
We show that the \textit{sample complexity of exploration} of our algorithm is bounded by $\tilde{O}({\frac{SA}{\epsilon^2(1-\gamma)^7}})$.
This improves the previously best known result of $\tilde{O}({\frac{SA}{\epsilon^4(1-\gamma)^8}})$ in this setting achieved by delayed Q-learning (Strehlet al., 2006),, and matches the lower bound in terms of $\epsilon$ as well as $S$ and $A$ up to logarithmic factors.

assistant: text: We adapt Q-learning with UCB-exploration bonus to infinite-horizon MDP with discounted rewards without accessing a generative model, and improves the previously best known result.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Progressive Growing of GANs for Improved Quality, Stability, and Variation

Paper content:
We describe a new training methodology for generative adversarial networks.
The key idea is to grow both the generator and discriminator progressively: starting from a low resolution, we add new layers that model increasingly fine details as training progresses.
This both speeds the training up and greatly stabilizes it, allowing us to produce images of unprecedented quality, e.g., CelebA images at 1024^2.
We also propose a simple way to increase the variation in generated images, and achieve a record inception score of 8.80 in unsupervised CIFAR10.
Additionally, we describe several implementation details that are important for discouraging unhealthy competition between the generator and discriminator.
Finally, we suggest a new metric for evaluating GAN results, both in terms of image quality and variation.
As an additional contribution, we construct a higher-quality version of the CelebA dataset.

assistant: text: We train generative adversarial networks in a progressive fashion, enabling us to generate high-resolution images with high quality.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: COMMUNITY PRESERVING NODE EMBEDDING

Paper content:
Detecting communities or the modular structure of real-life networks (e.g. a social
network or a product purchase network) is an important task because the way a
network functions is often determined by its communities.
The traditional approaches to community detection involve modularity-based approaches,
which generally speaking, construct partitions based on heuristics that
seek to maximize the ratio of the edges within the partitions to those between
them.
Node embedding approaches, which represent each node in a graph as a
real-valued vector, transform the problem of community detection in a graph to
that of clustering a set of vectors.
Existing node embedding approaches are primarily
based on first initiating uniform random walks from each node to construct
a context of a node and then seeks to make the vector representation of
the node close to its context.
However, standard node embedding approaches do
not directly take into account the community structure of a network while constructing
the context around each node.
To alleviate this, we explore two different
threads of work.
First, we investigate the use of biased random walks (specifically,
maximum entropy based walks) to obtain more centrality preserving embedding
of nodes, which we hypothesize may lead to more effective clusters in the embedded
space.
Second, we propose a community structure aware node embedding
approach where we incorporate modularity-based partitioning heuristics into
the objective function of node embedding.
We demonstrate that our proposed approach
for community detection outperforms a number of modularity-based baselines
as well as K-means on a standard node-embedded vector space (specifically,
node2vec) on a wide range of real-life networks of different sizes and densities.

assistant: text: A community preserving node embedding algorithm that results in more effective detection of communities with a clustering on the embedded space
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

Paper content:
Large-scale distributed training requires significant communication bandwidth for gradient exchange that limits the scalability of multi-node training, and requires expensive high-bandwidth network infrastructure.
The situation gets even worse with distributed training on mobile devices (federated learning), which suffers from higher latency, lower throughput, and intermittent poor connections.
In this paper, we find 99.9% of the gradient exchange in distributed SGD is redundant, and propose Deep Gradient Compression (DGC) to greatly reduce the communication bandwidth.
To preserve accuracy during compression, DGC employs four methods: momentum correction, local gradient clipping, momentum factor masking, and warm-up training.
We have applied Deep Gradient Compression to image classification, speech recognition, and language modeling with multiple datasets including Cifar10, ImageNet, Penn Treebank, and Librispeech Corpus.
On these scenarios, Deep Gradient Compression achieves a gradient compression ratio from 270x to 600x without losing accuracy, cutting the gradient size of ResNet-50 from 97MB to 0.35MB, and for DeepSpeech from 488MB to 0.74MB.
Deep gradient compression enables large-scale distributed training on inexpensive commodity 1Gbps Ethernet and facilitates distributed training on mobile.
Large-scale distributed training improves the productivity of training deeper and larger models BID7 BID35 BID24 BID37 .
Synchronous stochastic gradient descent (SGD) is widely used for distributed training.
By increasing the number of training nodes and taking advantage of data parallelism, the total computation time of the forward-backward passes on the same size training data can be dramatically reduced.
However, gradient exchange is costly and dwarfs the savings of computation time

assistant: text: we find 99.9% of the gradient exchange in distributed SGD is redundant; we reduce the communication bandwidth by two orders of magnitude without losing accuracy.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Few-Shot Regression via Learned Basis Functions

Paper content:
The recent rise in popularity of few-shot learning algorithms has enabled models to quickly adapt to new tasks based on only a few training samples.
Previous few-shot learning works have mainly focused on classification and reinforcement learning.
In this paper, we propose a few-shot meta-learning system that focuses exclusively on regression tasks.
Our model is based on the idea that the degree of freedom of the unknown function can be significantly reduced if it is represented as a linear combination of a set of appropriate basis functions.
This enables a few labelled samples to approximate the function.
We design a Feature Extractor network to encode basis functions for a task distribution, and a  Weights Generator to generate the weight vector for a novel task.
We show that our model outperforms the current state of the art meta-learning methods in various regression tasks.

assistant: text: This paper proposes a novel shot-learning method for small sample regression problems.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Unsupervised Domain Adaptation for Distance Metric Learning

Paper content:
Unsupervised domain adaptation is a promising avenue to enhance the performance of deep neural networks on a target domain, using labels only from a source domain.
However, the two predominant methods, domain discrepancy reduction learning and semi-supervised learning, are not readily applicable when source and target domains do not share a common label space.
This paper addresses the above scenario by learning a representation space that retains discriminative power on both the (labeled) source and (unlabeled) target domains while keeping representations for the two domains well-separated.
Inspired by a theoretical analysis, we first reformulate the disjoint classification task, where the source and target domains correspond to non-overlapping class labels, to a verification one.
To handle both within and cross domain verifications, we propose a Feature Transfer Network (FTN) to separate the target feature space from the original source space while aligned with a transformed source space.
Moreover, we present a non-parametric multi-class entropy minimization loss to further boost the discriminative power of FTNs on the target domain.
In experiments, we first illustrate how FTN works in a controlled setting of adapting from MNIST-M to MNIST with disjoint digit classes between the two domains and then demonstrate the effectiveness of FTNs through state-of-the-art performances on a cross-ethnicity face recognition problem.

assistant: text: Proposes a novel feature transfer network that optimizes domain adversarial loss and domain separation loss.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average

Paper content:
Presently the most successful approaches to semi-supervised learning are based on consistency regularization, whereby a model is trained to be robust to small perturbations of its inputs and parameters.
To understand consistency regularization, we conceptually explore how loss geometry interacts with training procedures.
The consistency loss dramatically improves generalization performance over supervised-only training; however, we show that SGD struggles to converge on the consistency loss and continues to make large steps that lead to changes in predictions on the test data.
Motivated by these observations, we propose to train consistency-based methods with Stochastic Weight Averaging (SWA), a recent approach which averages weights along the trajectory of SGD with a modified learning rate schedule.
We also propose fast-SWA, which further accelerates convergence by averaging multiple points within each cycle of a cyclical learning rate schedule.
With weight averaging, we achieve the best known semi-supervised results on CIFAR-10 and CIFAR-100, over many different quantities of labeled training data.
For example, we achieve 5.0% error on CIFAR-10 with only 4000 labels, compared to the previous best result in the literature of 6.3%.

assistant: text: The paper proposes to apply Stochastic Weight Averaging to the semi-supervised learning context, arguing that the semi-supervised MT/Pi models are especially amenable to SWA and propose fast SWA to speed up training.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Permutation Equivariant Models for Compositional Generalization in Language

Paper content:
Humans understand novel sentences by composing meanings and roles of core language components.
In contrast, neural network models for natural language modeling fail when such compositional generalization is required.
The main contribution of this paper is to hypothesize that language compositionality is a form of group-equivariance.
Based on this hypothesis, we propose a set of tools for constructing equivariant sequence-to-sequence models.
Throughout a variety of experiments on the SCAN tasks, we analyze the behavior of existing models under the lens of equivariance, and demonstrate that our equivariant architecture is able to achieve the type compositional generalization required in human language understanding.

assistant: text: We propose a link between permutation equivariance and compositional generalization, and provide equivariant language models
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Synthesizing realistic neural population activity patterns using Generative Adversarial Networks

Paper content:
The ability to synthesize realistic patterns of neural activity is crucial for studying neural information processing.
Here we used the Generative Adversarial Networks (GANs) framework to simulate the concerted activity of a population of neurons.
We adapted the Wasserstein-GAN variant to facilitate the generation of unconstrained neural population activity patterns while still benefiting from parameter sharing in the temporal domain.
We demonstrate that our proposed GAN, which we termed Spike-GAN, generates spike trains that match accurately the first- and second-order statistics of datasets of tens of neurons and also approximates well their higher-order statistics.
We applied Spike-GAN to a real dataset recorded from salamander retina and showed that it performs as well as state-of-the-art approaches based on the maximum entropy and the dichotomized Gaussian frameworks.
Importantly, Spike-GAN does not require to specify a priori the statistics to be matched by the model, and so constitutes a more flexible method than these alternative approaches.
Finally, we show how to exploit a trained Spike-GAN  to construct 'importance maps' to detect the most relevant statistical structures present in a spike train.
Spike-GAN provides a powerful, easy-to-use technique for generating realistic spiking neural activity and for describing the most relevant features of the large-scale neural population recordings studied in modern systems neuroscience.

assistant: text: The paper proposes to use GANs for synthesizing realistic neural activity patterns
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Not-So-Random Features

Paper content:
We propose a principled method for kernel learning, which relies on a Fourier-analytic characterization of translation-invariant or rotation-invariant kernels.
Our method produces a sequence of feature maps, iteratively refining the SVM margin.
We provide rigorous guarantees for optimality and generalization, interpreting our algorithm as online equilibrium-finding dynamics in a certain two-player min-max game.
Evaluations on synthetic and real-world datasets demonstrate scalability and consistent improvements over related random features-based methods.

assistant: text: The paper proposes to learn a custom translation or rotation invariant kernal in the Fourier representation to maximize the margin of SVM.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Multi-task Learning on MNIST Image Datasets

Paper content:
We apply multi-task learning to image classification tasks on MNIST-like datasets.
MNIST dataset has been referred to as the {\em drosophila} of machine learning and has been the testbed of many learning theories.
The NotMNIST dataset and the FashionMNIST dataset have been created with the MNIST dataset as reference.
In this work, we exploit these MNIST-like datasets for multi-task learning.
The datasets are pooled together for learning the parameters of joint classification networks.
Then the learned parameters are used as the initial parameters to retrain disjoint classification networks.
The baseline recognition model are all-convolution neural networks.
Without multi-task learning, the recognition accuracies for MNIST, NotMNIST and FashionMNIST are 99.56\%, 97.22\% and 94.32\% respectively.
With multi-task learning to pre-train the networks, the recognition accuracies are respectively 99.70\%, 97.46\% and 95.25\%.
The results re-affirm that multi-task learning framework, even with data with different genres, does lead to significant improvement.

assistant: text: multi-task learning works
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Disentanglement with Hyperspherical Latent Spaces using Diffusion Variational Autoencoders

Paper content:
A disentangled representation of a data set should be capable of recovering the underlying factors that generated it.
One question that arises is whether using Euclidean space for latent variable models can produce a disentangled representation when the underlying generating factors have a certain geometrical structure.
Take for example the images of a car seen from different angles.
The angle has a periodic structure but a 1-dimensional representation would fail to capture this topology.
How can we address this problem?
The submissions presented for the first stage of the  NeurIPS2019 Disentanglement Challenge consist of a Diffusion Variational Autoencoder ($\Delta$VAE) with a hyperspherical latent space which can for example recover periodic true factors.
The training of the $\Delta$VAE is enhanced by incorporating a modified version of the Evidence Lower Bound (ELBO) for tailoring the encoding capacity of the posterior approximate.

assistant: text: Description of submission to NeurIPS2019 Disentanglement Challenge based on hyperspherical variational autoencoders
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Automatically Inferring Data Quality for Spatiotemporal Forecasting

Paper content:
Spatiotemporal forecasting has become an increasingly important prediction task in machine learning and statistics due to its vast applications, such as climate modeling, traffic prediction, video caching predictions, and so on.
While numerous studies have been conducted, most existing works assume that the data from different sources or across different locations are equally reliable.
Due to cost, accessibility, or other factors, it is inevitable that the data quality could vary, which introduces significant biases into the model and leads to unreliable prediction results.
The problem could be exacerbated in black-box prediction models, such as deep neural networks.
In this paper, we propose a novel solution that can automatically infer data quality levels of different sources through local variations of spatiotemporal signals without explicit labels.
Furthermore, we integrate the estimate of data quality level with graph convolutional networks to exploit their efficient structures.
We evaluate our proposed method on forecasting temperatures in Los Angeles.

assistant: text: This work proposed a new way to evaluate the quality of different data sources with the time-vary graph model, with the quality level used as a regularization term in the objective function
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Discrete flow posteriors for variational inference in discrete dynamical systems

Paper content:
Each training step for a variational autoencoder (VAE) requires us to sample from the approximate posterior, so we usually choose simple (e.g. factorised) approximate posteriors in which sampling is an efficient computation that fully exploits GPU parallelism.
However, such simple approximate posteriors are often insufficient, as they eliminate statistical dependencies in the posterior.
While it is possible to use normalizing flow approximate posteriors for continuous latents, there is nothing analogous for discrete latents.
The most natural approach to model discrete dependencies is an autoregressive distribution, but sampling from such distributions is inherently sequential and thus slow.
We develop a fast, parallel sampling procedure for autoregressive distributions based on fixed-point iterations which enables efficient and accurate variational inference in discrete state-space models.
To optimize the variational bound, we considered two ways to evaluate probabilities: inserting the relaxed samples directly into the pmf for the discrete distribution, or converting to continuous logistic latent variables and interpreting the K-step fixed-point iterations as a normalizing flow.
We found that converting to continuous latent variables gave considerable additional scope for mismatch between the true and approximate posteriors, which resulted in biased inferences, we thus used the former approach.
We tested our approach on the neuroscience problem of inferring discrete spiking activity from noisy calcium-imaging data, and found that it gave accurate connectivity estimates in an order of magnitude less time.

assistant: text: The authors posit a general autoregressive posterior family for discrete variables or their continuous relaxations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Modelling the influence of data structure on learning in neural networks

Paper content:
The lack of crisp mathematical models that capture the structure of real-world
data sets is a major obstacle to the detailed theoretical understanding of deep
neural networks.
Here, we first demonstrate the effect of structured data sets
by experimentally comparing the dynamics and the performance of two-layer
networks trained on two different data sets:
(i) an unstructured synthetic data
set containing random i.i.d. inputs, and
(ii) a simple canonical data set such
as MNIST images.
Our analysis reveals two phenomena related to the dynamics of
the networks and their ability to generalise that only appear when training on
structured data sets.
Second, we introduce a generative model for data sets,
where high-dimensional inputs lie on a lower-dimensional manifold and have
labels that depend only on their position within this manifold.
We call it the
*hidden manifold model* and we experimentally demonstrate that training
networks on data sets drawn from this model reproduces both the phenomena seen
during training on MNIST.

assistant: text: We demonstrate how structure in data sets impacts neural networks and introduce a generative model for synthetic data sets that reproduces this impact.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks

Paper content:
Learning when to communicate and doing that effectively is essential in multi-agent tasks.
Recent works show that continuous communication allows efficient training with back-propagation in multi-agent scenarios, but have been restricted to fully-cooperative tasks.
In this paper, we present Individualized Controlled Continuous Communication Model (IC3Net) which has better training efficiency than simple continuous communication model, and can be applied to semi-cooperative and competitive settings along with the cooperative settings.
IC3Net controls continuous communication with a gating mechanism and uses individualized rewards foreach agent to gain better performance and scalability while fixing credit assignment issues.
Using variety of tasks including StarCraft BroodWars explore and combat scenarios, we show that our network yields improved performance and convergence rates than the baselines as the scale increases.
Our results convey that IC3Net agents learn when to communicate based on the scenario and profitability.

assistant: text: We introduce IC3Net, a single network which can be used to train agents in cooperative, competitive and mixed scenarios. We also show that agents can learn when to communicate using our model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Reinforcement Learning with Chromatic Networks

Paper content:
We present a neural architecture search algorithm to construct compact reinforcement learning (RL) policies, by combining ENAS and ES in a highly scalable and intuitive way.
By defining the combinatorial search space of NAS to be the set of different edge-partitionings (colorings) into same-weight classes, we represent compact architectures via efficient learned edge-partitionings.
For several RL tasks, we manage to learn colorings translating to effective policies parameterized by as few as 17 weight parameters, providing >90 % compression over vanilla policies and 6x compression over state-of-the-art compact policies based on Toeplitz matrices, while still maintaining good reward.
We believe that our work is one of the first attempts to propose a rigorous approach to training structured neural network architectures for RL problems that are of interest especially in mobile robotics with limited storage and computational resources.

assistant: text: The authors construct reinforcement learning policies with very few parameters by compressing a feed-forward neural network, forcing it to share weights, and using a reinforcement learning method to learn the mapping of shared weights.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Context Dependent Modulation of Activation Function

Paper content:
We propose a modification to traditional Artificial Neural Networks (ANNs), which provides the ANNs with new aptitudes motivated by biological neurons.
Biological neurons work far beyond linearly summing up synaptic inputs and then transforming the integrated information.
A biological neuron change firing modes accordingly to peripheral factors (e.g., neuromodulators) as well as intrinsic ones.
Our modification connects a new type of ANN nodes, which mimic the function of biological neuromodulators and are termed modulators, to enable other traditional ANN nodes to adjust their activation sensitivities in run-time based on their input patterns.
In this manner, we enable the slope of the activation function to be context dependent.
This modification produces statistically significant improvements in comparison with traditional ANN nodes in the context of Convolutional Neural Networks and Long Short-Term Memory networks.

assistant: text: Introduction of an architectural change for basic neurons in a neural network, and the idea to multiply neuron linear combination output by a modulator prior to feeding it into the activation function.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Detecting Topological Defects in 2D Active Nematics Using Convolutional Neural Networks

Paper content:
Active matter consists of active agents which transform energy extracted from surroundings into momentum, producing a variety of collective phenomena.
A model, synthetic active system composed of microtubule polymers driven by protein motors spontaneously forms a liquid-crystalline nematic phase.
Extensile stress created by the protein motors precipitates continuous buckling and folding of the microtubules creating motile topological defects and turbulent fluid flows.
Defect motion is determined by the rheological properties of the material; however, these remain largely unquantified.
Measuring defects dynamics can yield fundamental insights into active nematics, a class of materials that include bacterial films and animal cells.
Current methods for defect detection lack robustness and precision, and require fine-tuning for datasets with different visual quality.
In this study, we applied Deep Learning to train a defect detector to automatically analyze microscopy videos of the microtubule active nematic.
Experimental results indicate that our method is robust and accurate.
It is expected to significantly increase the amount of video data that can be processed.

assistant: text: The authors demonstrate that a deep learning approach offers improvement to both the identification accuracy and rate at which defects can be identified of nematic liquid crystals.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Biologically-Inspired Spatial Neural Networks

Paper content:
We introduce bio-inspired artificial neural networks consisting of neurons that are additionally characterized by spatial positions.
To simulate properties of biological systems we add the costs penalizing long connections and the proximity of neurons in a two-dimensional space.
Our experiments show that in the case where the network performs two different tasks, the neurons naturally split into clusters, where each cluster is responsible for processing a different task.
This behavior not only corresponds to the biological systems, but also allows for further insight into interpretability or continual learning.

assistant: text: Bio-inspired artificial neural networks, consisting of neurons positioned in a two-dimensional space, are capable of forming independent groups for performing different tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework

Paper content:
Learning multilingual representations of text has proven a successful method for many cross-lingual transfer learning tasks.
There are two main paradigms for learning such representations: (1) alignment, which maps different independently trained monolingual representations into a shared space, and (2) joint training, which directly learns unified multilingual representations using monolingual and cross-lingual objectives jointly.
In this paper, we first conduct direct comparisons of representations learned using both of these methods across diverse cross-lingual tasks.
Our empirical results reveal a set of pros and cons for both methods, and show that the relative performance of alignment versus joint training is task-dependent.
Stemming from this analysis, we propose a simple and novel framework that combines these two previously mutually-exclusive approaches.
Extensive experiments on various tasks demonstrate that our proposed framework alleviates limitations of both approaches, and outperforms existing methods on the MUSE bilingual lexicon induction (BLI) benchmark.
We further show that our proposed framework can generalize to contextualized representations and achieves state-of-the-art results on the CoNLL cross-lingual NER benchmark.

assistant: text: We conduct a comparative study of cross-lingual alignment vs joint training methods and unify these two previously exclusive paradigms in a new framework.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Address2vec: Generating vector embeddings for blockchain analytics

Paper content:
Bitcoin is a virtual coinage system that enables users to trade virtually free of a central trusted authority.
All transactions on the Bitcoin blockchain are publicly available for viewing, yet as Bitcoin is built mainly for security it’s original structure does not allow for direct analysis of address transactions.
Existing analysis methods of the Bitcoin blockchain can be complicated, computationally expensive or inaccurate.
We propose a computationally efficient model to analyze bitcoin blockchain addresses and allow for their use with existing machine learning algorithms.
We compare our approach against Multi Level Sequence Learners (MLSLs), one of the best performing models on bitcoin address data.

assistant: text: The paper proposes to use an autoencoder, networkX, and node2Vec to predict whether a Bitcoin address will become empty after a year, but the results are worse than an existing baseline.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: ODE Analysis of Stochastic Gradient Methods with Optimism and Anchoring  for Minimax Problems and GANs

Paper content:
Despite remarkable empirical success, the training dynamics of generative adversarial networks (GAN), which involves solving a minimax game using stochastic gradients, is still poorly understood.
In this work, we analyze last-iterate convergence of simultaneous gradient descent (simGD) and its variants under the assumption of convex-concavity, guided by a continuous-time analysis with differential equations.
First, we show that simGD, as is, converges with stochastic sub-gradients under strict convexity in the primal variable.
Second, we generalize optimistic simGD to accommodate an optimism rate separate from the learning rate and show its convergence with full gradients.
Finally, we present anchored simGD, a new method, and show convergence with stochastic subgradients.

assistant: text: An anaysis of simultaneous stochastic subgradient, simultaneous gradient with optimism, and simultaneous gradient with anchoring in the context of minmax convex concave games.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Fluctuation-dissipation relations for stochastic gradient descent

Paper content:
The notion of the stationary equilibrium ensemble has played a central role in statistical mechanics.
In machine learning as well, training serves as generalized equilibration that drives the probability distribution of model parameters toward stationarity.
Here, we derive stationary fluctuation-dissipation relations that link measurable quantities and hyperparameters in the stochastic gradient descent algorithm.
These relations hold exactly for any stationary state and can in particular be used to adaptively set training schedule.
We can further use the relations to efficiently extract information pertaining to a loss-function landscape such as the magnitudes of its Hessian and anharmonicity.
Our claims are empirically verified.

assistant: text: The authors derive the stationary fluctuation-dissipation relations that link measurable quantities and hyperparameters in SGD and use the relations to set training schedule adaptively and analyze the loss-function landscape.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step

Paper content:
Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion.
Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost.
GANs are designed to reach a Nash equilibrium at which each player cannot reduce their cost without changing the other players’ parameters.
One useful approach for the theory of GANs is to show that a divergence between the training distribution and the model distribution obtains its minimum value at equilibrium.
Several recent research directions have been motivated by the idea that this divergence is the primary guide for the learning process and that every step of learning should decrease the divergence.
We show that this view is overly restrictive.
During GAN training, the discriminator provides learning signal in situations where the gradients of the divergences between distributions would not be useful.
We provide empirical counterexamples to the view of GAN training as divergence minimization.
Specifically, we demonstrate that GANs are able to learn distributions in situations where the divergence minimization point of view predicts they would fail.
We also show that gradient penalties motivated from the divergence minimization perspective are equally helpful when applied in other contexts in which the divergence minimization perspective does not predict they would be helpful.
This contributes to a growing body of evidence that GAN training may be more usefully viewed as approaching Nash equilibria via trajectories that do not necessarily minimize a specific divergence at each step.

assistant: text: This paper studies non-saturating GANs and the effect of two penalized gradient approaches, considering several thought experiments to demonstrate observations and validate them on real data experiments.