ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Numerical Attributes in Knowledge Bases

Paper content:
Knowledge bases (KB) are often represented as a collection of facts in the form (HEAD, PREDICATE, TAIL), where HEAD and TAIL are entities while PREDICATE is a binary relationship that links the two.
It is a well-known fact that knowledge bases are far from complete, and hence the plethora of research on KB completion methods, specifically on link prediction.
However, though frequently ignored, these repositories also contain numerical facts.
Numerical facts link entities to numerical values via numerical predicates; e.g., (PARIS, LATITUDE, 48.8).
Likewise, numerical facts also suffer from the incompleteness problem.
To address this issue, we introduce the numerical attribute prediction problem.
This problem involves a new type of query where the relationship is a numerical predicate.
Consequently, and contrary to link prediction, the answer to this query is a numerical value.
We argue that the numerical values associated with entities explain, to some extent, the relational structure of the knowledge base.
Therefore, we leverage knowledge base embedding methods to learn representations that are useful predictors for the numerical attributes.
An extensive set of experiments on benchmark versions of FREEBASE and YAGO show that our approaches largely outperform sensible baselines.
We make the datasets available under a permissive BSD-3 license.

assistant: text: Prediction of numerical attribute values associated with entities in knowledge bases.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Gating out sensory noise in a spike-based Long Short-Term Memory network

Paper content:
Spiking neural networks are being investigated both as biologically plausible models of neural computation and also as a potentially more efficient type of neural network.
While convolutional spiking neural networks have been demonstrated to achieve near state-of-the-art performance, only one solution has been proposed to convert gated recurrent neural networks, so far.
Recurrent neural networks in the form of networks of gating memory cells have been central in state-of-the-art solutions in problem domains that involve sequence recognition or generation.
Here, we design an analog gated LSTM cell where its neurons can be substituted for efficient stochastic spiking neurons.
These adaptive spiking neurons implement an adaptive form of sigma-delta coding to convert internally computed analog activation values to spike-trains.
For such neurons, we approximate the effective activation function, which resembles a sigmoid.
We show how analog neurons with such activation functions can be used to create an analog LSTM cell; networks of these cells can then be trained with standard backpropagation.
We train these LSTM networks on a noisy and noiseless version of the original sequence prediction task from Hochreiter & Schmidhuber (1997), and also on a noisy and noiseless version of a classical working memory reinforcement learning task, the T-Maze.
Substituting the analog neurons for corresponding adaptive spiking neurons, we then show that almost all resulting spiking neural network equivalents correctly compute the original tasks.

assistant: text: We demonstrate a gated recurrent asynchronous spiking neural network that corresponds to an LSTM unit.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Directional Analysis of Stochastic Gradient Descent via von Mises-Fisher Distributions in Deep Learning

Paper content:
Although stochastic gradient descent (SGD) is a driving force behind the recent success of deep learning, our understanding of its dynamics in a high-dimensional parameter space is limited.
In recent years, some researchers have used the stochasticity of minibatch gradients, or the signal-to-noise ratio, to better characterize the learning dynamics of SGD.
Inspired from these work, we here analyze SGD from a geometrical perspective by inspecting the stochasticity of the norms and directions of minibatch gradients.
We propose a model of the directional concentration for minibatch gradients through von Mises-Fisher (VMF) distribution, and show that the directional uniformity of minibatch gradients increases over the course of SGD.
We empirically verify our result using deep convolutional networks and observe a higher correlation between the gradient stochasticity and the proposed directional uniformity than that against the gradient norm stochasticity, suggesting that the directional statistics of minibatch gradients is a major factor behind SGD.

assistant: text: One of theoretical issues in deep learning
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Deep Generative Acoustic Model for Compositional Automatic Speech Recognition

Paper content:
Inspired by the recent successes of deep generative models for Text-To-Speech (TTS) such as WaveNet (van den Oord et al., 2016) and Tacotron (Wang et al., 2017), this article proposes the use of a deep generative model tailored for Automatic Speech Recognition (ASR) as the primary acoustic model (AM) for an overall recognition system with a separate language model (LM).
Two dimensions of depth are considered: (1) the use of mixture density networks, both autoregressive and non-autoregressive, to generate density functions capable of modeling acoustic input sequences with much more powerful conditioning than the first-generation generative models for ASR, Gaussian Mixture Models / Hidden Markov Models (GMM/HMMs), and (2) the use of standard LSTMs, in the spirit of the original tandem approach, to produce discriminative feature vectors for generative modeling.
Combining mixture density networks and deep discriminative features leads to a novel dual-stack LSTM architecture directly related to the RNN Transducer (Graves, 2012), but with the explicit functional form of a density, and combining naturally with a separate language model, using Bayes rule.
The generative models discussed here are compared experimentally in terms of log-likelihoods and frame accuracies.

assistant: text: This paper proposes the use of a deep generative acoustic model for automatic speech recognition, combining naturally with other deep sequence-to-sequence modules using Bayes' rule.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generalized Natural Language Grounded Navigation via Environment-agnostic Multitask Learning

Paper content:
Recent research efforts enable study for natural language grounded navigation in photo-realistic environments, e.g., following natural language instructions or dialog.
However, existing methods tend to overfit training data in seen environments and fail to generalize well in previously unseen environments.
In order to close the gap between seen and unseen environments, we aim at learning a generalizable navigation model from two novel perspectives:
(1) we introduce a multitask navigation model that can be seamlessly trained on both Vision-Language Navigation (VLN) and Navigation from Dialog History (NDH) tasks, which benefits from richer natural language guidance and effectively transfers knowledge across tasks;
(2) we propose to learn environment-agnostic representations for navigation policy that are invariant among environments, thus generalizing better on unseen environments.
Extensive experiments show that our environment-agnostic multitask navigation model significantly reduces the performance gap between seen and unseen environments and outperforms the baselines on unseen environments by 16% (relative measure on success rate) on VLN and 120% (goal progress) on NDH, establishing the new state of the art for NDH task.

assistant: text: We propose to learn a more generalized policy for natural language grounded navigation tasks via environment-agnostic multitask learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: ADef: an Iterative Algorithm to Construct Adversarial Deformations

Paper content:
While deep neural networks have proven to be a powerful tool for many recognition and classification tasks, their stability properties are still not well understood.
In the past, image classifiers have been shown to be vulnerable to so-called adversarial attacks, which are created by additively perturbing the correctly classified image.
In this paper, we propose the ADef algorithm to construct a different kind of adversarial attack created by iteratively applying small deformations to the image, found through a gradient descent step.
We demonstrate our results on MNIST with convolutional neural networks and on ImageNet with Inception-v3 and ResNet-101.

assistant: text: We propose a new, efficient algorithm to construct adversarial examples by means of deformations, rather than additive perturbations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Robust training with ensemble consensus

Paper content:
Since deep neural networks are over-parameterized, they can memorize noisy examples.
We address such memorizing issue in the presence of annotation noise.
From the fact that deep neural networks cannot generalize neighborhoods of the features acquired via memorization, we hypothesize that noisy examples do not consistently incur small losses on the network under a certain perturbation.
Based on this, we propose a novel training method called Learning with Ensemble Consensus (LEC) that prevents overfitting noisy examples by eliminating them using the consensus of an ensemble of perturbed networks.
One of the proposed LECs, LTEC outperforms the current state-of-the-art methods on noisy MNIST, CIFAR-10, and CIFAR-100 in an efficient manner.

assistant: text: This work presents a method of generating and using ensembles effectively to identify noisy examples in the presence of annotation noise.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Line attractor dynamics in recurrent networks for sentiment classiﬁcation

Paper content:
Recurrent  neural  networks  (RNNs)  are  a  powerful tool for modeling sequential data.
Despite their widespread usage, understanding how RNNs solve complex problems remains elusive.
Here, we characterize how popular RNN architectures perform document-level sentiment classification.
Despite their theoretical capacity to implement complex, high-dimensional computations, we find that  trained  networks  converge  to  highly  interpretable, low-dimensional representations.
We identify a simple mechanism, integration along an approximate line attractor, and find this mechanism present across RNN architectures (including LSTMs, GRUs, and vanilla RNNs).
Overall, these results demonstrate that surprisingly universal and human interpretable computations can arise across a range of recurrent networks.

assistant: text: We analyze recurrent networks trained on sentiment classification, and find that they all exhibit approximate line attractor dynamics when solving this task.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A CASE STUDY ON OPTIMAL DEEP LEARNING MODEL FOR UAVS

Paper content:
Over the passage of time Unmanned Autonomous Vehicles (UAVs), especially
Autonomous flying drones grabbed a lot of attention in Artificial Intelligence.
Since electronic technology is getting smaller, cheaper and more efficient, huge
advancement in the study of UAVs has been observed recently.
From monitoring
floods, discerning the spread of algae in water bodies to detecting forest trail, their
application is far and wide.
Our work is mainly focused on autonomous flying
drones where we establish a case study towards efficiency, robustness and accuracy
of UAVs where we showed our results well supported through experiments.
We provide details of the software and hardware architecture used in the study.
We
further discuss about our implementation algorithms and present experiments that
provide a comparison between three different state-of-the-art algorithms namely
TrailNet, InceptionResnet and MobileNet in terms of accuracy, robustness, power
consumption and inference time.
In our study, we have shown that MobileNet has
produced better results with very less computational requirement and power consumption.
We have also reported the challenges we have faced during our work
as well as a brief discussion on our future work to improve safety features and
performance.

assistant: text: case study on optimal deep learning model for UAVs
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Unsupervised Demixing of Structured Signals from Their Superposition Using GANs

Paper content:
Recently, Generative Adversarial Networks (GANs) have emerged as a popular alternative for modeling complex high dimensional distributions.
Most of the existing works implicitly assume that the clean samples from the target distribution are easily available.
However, in many applications, this assumption is violated.
In this paper, we consider the observation setting in which the samples from a target distribution are given by the superposition of two structured components, and leverage GANs for learning of the structure of the components.
We propose a novel framework, demixing-GAN, which learns the distribution of two components at the same time.
Through extensive numerical experiments, we demonstrate that the proposed framework can generate clean samples from unknown distributions, which further can be used in demixing of the unseen test images.

assistant: text: An unsupervised learning approach for separating two structured signals from their superposition
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learned imaging with constraints and uncertainty quantification

Paper content:
We outline new approaches to incorporate ideas from deep learning into wave-based least-squares imaging.
The aim, and main contribution of this work, is the combination of handcrafted constraints with deep convolutional neural networks, as a way to harness their remarkable ease of generating natural images.
The mathematical basis underlying our method is the expectation-maximization framework, where data are divided in batches and coupled to additional "latent" unknowns.
These unknowns are pairs of elements from the original unknown space (but now coupled to a specific data batch) and network inputs.
In this setting, the neural network controls the similarity between these additional parameters, acting as a "center" variable.
The resulting problem amounts to a maximum-likelihood estimation of the network parameters when the augmented data model is marginalized over the latent variables.
In this work, we tested an inverse problem framework which includes hard constraints and deep priors.
Hard constraints are necessary in many problems, such as seismic imaging, where the unknowns must belong to a feasible set in order to ensure the numerical stability of the forward problem.
Deep priors, enforced through adherence to the range of a neural network, provide an additional, implicit type of regularization, as demonstrated by recent work [2, Dittmer et al. [3] ], and corroborated by our numerical results.
The resulting algorithm can be mathematically interpreted in light of expectation maximization methods.
Furthermore, connections to elastic averaging SGD [10] highlight potential computational benefits of a parallel (synchronous or asynchronous) implementation.
On a speculative note, we argue that the presented method, which combines stochastic optimization on the dual variable with on-the-fly estimation of the generative model's weights using Langevin dynamics, reaps information on the "posterior" distribution leveraging multiplicity in the data and the fact that the data is acquired over one and the same Earth model.
Our preliminary results seem consistent with a behavior to be expected from a "posterior" distribution.
b,c) sample "prior" (before training) and "posterior" distribution functions for two points in the model.

assistant: text: We combine hard handcrafted constraints with a deep prior weak constraint to perform seismic imaging and reap information on the "posterior" distribution leveraging multiplicity in the data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On recognition of Cyrillic Text

Paper content:
We introduce the largest (among publicly available) dataset for Cyrillic Handwritten Text Recognition and the first dataset for Cyrillic Text in the Wild Recognition, as well as suggest a method for recognizing Cyrillic Handwritten Text and Text in the Wild.
Based on this approach, we develop a system that can reduce the document processing time for one of the largest mathematical competitions in Ukraine by 12 days and the amount of used paper by 0.5 ton.
Text is one of the main ways to transfer the information and it can take many forms.
It can be handwritten or printed, in the form of business documents, notes, bills, historical documents, advertisements, logos etc.
Therefore, the method for its recognition should be flexible enough to work with different text styles and under the different external conditions.
Although for the English language the task of text recognition is well studied [1] , [2] , for Cyrillic languages such studies are almost missing, the main reason being the lack of extensive publicly available datasets.
To the best of our knowledge, the only public Cyrillic dataset consists only of individual letters [3] , while others [1] , [4] , [5] , are unavailable.
In our research, we will focus on developing a single model for Handwritten Text Recognition and Text Recognition in the Wild, as the extreme case of printed text.

assistant: text: We introduce several datasets for Cyrillic OCR and a method for its recognition
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Luck Matters: Understanding Training Dynamics of Deep ReLU Networks

Paper content:
We analyze the dynamics of training deep ReLU networks and their implications on generalization capability.
Using a teacher-student setting, we discovered a novel relationship between the gradient received by hidden student nodes and the activations of teacher nodes for deep ReLU networks.
With this relationship and the assumption of small overlapping teacher node activations, we prove that (1) student nodes whose weights are initialized to be close to teacher nodes converge to them at a faster rate, and (2) in over-parameterized regimes and 2-layer case, while a small set of lucky nodes do converge to the teacher nodes, the fan-out weights of other nodes converge to zero.
This framework provides insight into multiple puzzling phenomena in deep learning like over-parameterization, implicit regularization, lottery tickets, etc.
We verify our assumption by showing that the majority of BatchNorm biases of pre-trained VGG11/16 models are negative.
Experiments on (1) random deep teacher networks with Gaussian inputs, (2) teacher network pre-trained on CIFAR-10 and (3) extensive ablation studies validate our multiple theoretical predictions.

assistant: text: A theoretical framework for deep ReLU network that can explains multiple puzzling phenomena like over-parameterization, implicit regularization, lottery tickets, etc.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Directional Message Passing for Molecular Graphs

Paper content:
Graph neural networks have recently achieved great successes in predicting quantum mechanical properties of molecules.
These models represent a molecule as a graph using only the distance between atoms (nodes) and not the spatial direction from one atom to another.
However, directional information plays a central role in empirical potentials for molecules, e.g. in angular potentials.
To alleviate this limitation we propose directional message passing, in which we embed the messages passed between atoms instead of the atoms themselves.
Each message is associated with a direction in coordinate space.
These directional message embeddings are rotationally equivariant since the associated directions rotate with the molecule.
We propose a message passing scheme analogous to belief propagation, which uses the directional information by transforming messages based on the angle between them.
Additionally, we use spherical Bessel functions to construct a theoretically well-founded, orthogonal radial basis that achieves better performance than the currently prevalent Gaussian radial basis functions while using more than 4x fewer parameters.
We leverage these innovations to construct the directional message passing neural network (DimeNet).
DimeNet outperforms previous GNNs on average by 77% on MD17 and by 41% on QM9.

assistant: text: Directional message passing incorporates spatial directional information to improve graph neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Classification in the dark using tactile exploration

Paper content:
Combining information from different sensory modalities to execute goal directed actions is a key aspect of human intelligence.
Specifically, human agents are very easily able to translate the task communicated in one sensory domain (say vision) into a representation that enables them to complete this task when they can only sense their environment using a separate sensory modality (say touch).
In order to build agents with similar capabilities, in this work we consider the problem of a retrieving a target object from a drawer.
The agent is provided with an image of a previously unseen object and it explores objects in the drawer using only tactile sensing to retrieve the object that was shown in the image without receiving any visual feedback.
Success at this task requires close integration of visual and tactile sensing.
We present a method for performing this task in a simulated environment using an anthropomorphic hand.
We hope that future research in the direction of combining sensory signals for acting will find the object retrieval from a drawer to be a useful benchmark problem

assistant: text: In this work, we study the problem of learning representations to identify novel objects by exploring objects using tactile sensing. Key point here is that the query is provided in image domain.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Enabling Limited Resource-Bounded Disjunction in Scheduling

Paper content:
We describe three approaches to enabling an extremely computationally limited embedded scheduler to consider a small number of alternative activities based on resource availability.
We consider the case where the scheduler is so computationally limited that it cannot backtrack search.
The first two approaches precompile resource checks (called guards) that only enable selection of a preferred alternative activity if sufficient resources are estimated to be available to schedule the remaining activities.
The final approach mimics backtracking by invoking the scheduler multiple times with the alternative activities.
We present an evaluation of these techniques on mission scenarios (called sol types) from NASA's next planetary rover where these techniques are being evaluated for inclusion in an onboard scheduler.

assistant: text: This paper describes three techniques to allow a non-backtracking,  computationally limited scheduler to consider a small number of alternative activities based on resource availability.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: ATLPA:ADVERSARIAL TOLERANT LOGIT PAIRING WITH ATTENTION FOR CONVOLUTIONAL NEURAL NETWORK

Paper content:
Though deep neural networks have achieved the state of the art performance in visual classification, recent studies have shown that they are all vulnerable to the attack of adversarial examples.
To solve the problem, some regularization adversarial training methods, constraining the output label or logit, have been studied.
In this paper, we propose a novel regularized adversarial training framework ATLPA,namely Adversarial Tolerant Logit Pairing with Attention.
Instead of constraining a hard distribution (e.g., one-hot vectors or logit) in adversarial training, ATLPA uses Tolerant Logit which consists of confidence distribution on top-k classes and captures inter-class similarities at the image level.
Specifically, in addition to minimizing the empirical loss, ATLPA encourages attention map for pairs of examples to be similar.
When applied to clean examples and their adversarial counterparts, ATLPA improves accuracy on adversarial examples over adversarial training.
We evaluate ATLPA with the state of the art algorithms, the experiment results show that our method outperforms these baselines with higher accuracy.
Compared with previous work, our work is evaluated under highly challenging PGD attack: the maximum perturbation $\epsilon$ is 64 and 128 with 10 to 200 attack iterations.

assistant: text: In this paper, we propose a novel regularized adversarial training framework ATLPA,namely Adversarial Tolerant Logit Pairing with Attention.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On the effect of the activation function on the distribution of hidden nodes in a deep network

Paper content:
We analyze the joint probability distribution on the lengths of the
vectors of hidden variables in different layers of a fully connected
deep network, when the weights and biases are chosen randomly according to
Gaussian distributions, and the input is binary-valued.
We show
that, if the activation function satisfies a minimal set of
assumptions, satisfied by all activation functions that we know that
are used in practice, then, as the width of the network gets large,
the ``length process'' converges in probability to a length map
that is determined as a simple function of the variances of the
random weights and biases, and the activation function.
We also show that this convergence may fail for activation functions
that violate our assumptions.

assistant: text: We prove that, for activation functions satisfying some conditions, as a deep network gets wide, the lengths of the vectors of hidden variables converge to a length map.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Bayesian Time Series Forecasting with Change Point and Anomaly Detection

Paper content:
Time series forecasting plays a crucial role in marketing, finance and many other quantitative fields.
A large amount of methodologies has been developed on this topic, including ARIMA, Holt–Winters, etc.
However, their performance is easily undermined by the existence of change points and anomaly points, two structures commonly observed in real data, but rarely considered in the aforementioned methods.
In this paper, we propose a novel state space time series model, with the capability to capture the structure of change points and anomaly points, as well as trend and seasonality.
To infer all the hidden variables, we develop a Bayesian framework, which is able to obtain distributions and forecasting intervals for time series forecasting, with provable theoretical properties.
For implementation, an iterative algorithm with Markov chain Monte Carlo (MCMC), Kalman filter and Kalman smoothing is proposed.
In both synthetic data and real data applications, our methodology yields a better performance in time series forecasting compared with existing methods, along with more accurate change point detection and anomaly detection.

assistant: text: We propose a novel state space time series model with the capability to capture the structure of change points and anomaly points, so that it has a better forecasting performance when there exist change points and anomalies in the time series.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient and Robust Asynchronous Federated Learning with Stragglers

Paper content:
We address the efficiency issues caused by the straggler effect in the recently emerged federated learning, which collaboratively trains a model on decentralized non-i.i.d. (non-independent and identically distributed) data across massive worker devices without exchanging training data in the unreliable and heterogeneous networks.
We propose a novel two-stage analysis on the error bounds of general federated learning, which provides practical insights into optimization.
As a result, we propose a novel easy-to-implement federated learning algorithm that uses asynchronous settings and strategies to control discrepancies between the global model and delayed models and adjust the number of local epochs with the estimation of staleness to accelerate convergence and resist performance deterioration caused by stragglers.
Experiment results show that our algorithm converges fast and robust on the existence of massive stragglers.

assistant: text: We propose an efficient and robust asynchronous federated learning algorithm on the existence of stragglers
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Temporal Abstraction with Information-theoretic Constraints for Hierarchical Reinforcement Learning

Paper content:
Applying reinforcement learning (RL) to real-world problems will require reasoning about action-reward correlation over long time horizons.
Hierarchical reinforcement learning (HRL) methods handle this by dividing the task into hierarchies, often with hand-tuned network structure or pre-defined subgoals.
We propose a novel HRL framework TAIC, which learns the temporal abstraction from past experience or expert demonstrations without task-specific knowledge.
We formulate the temporal abstraction problem as learning latent representations of action sequences and present a novel approach of regularizing the latent space by adding information-theoretic constraints.
Specifically, we maximize the mutual information between the latent variables and the state changes.
A visualization of the latent space demonstrates that our algorithm learns an effective abstraction of the long action sequences.
The learned abstraction allows us to learn new tasks on higher level more efficiently.
We convey a significant speedup in convergence over benchmark learning problems.
These results demonstrate that learning temporal abstractions is an effective technique in increasing the convergence rate and sample efficiency of RL algorithms.

assistant: text: We propose a novel HRL framework, in which we formulate the temporal abstraction problem as learning a latent representation  of  action  sequence.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: AuthAR: Concurrent Authoring of Tutorials for AR Assembly Guidance

Paper content:
Augmented Reality (AR) can assist with physical tasks such as object assembly through the use of “situated instructions”.
These instructions can be in the form of videos, pictures, text or guiding animations, where the most helpful media among these is highly dependent on both the user and the nature of the task.
Our work supports the authoring of AR tutorials for assembly tasks with little overhead beyond simply performing the task itself.
The presented system, AuthAR reduces the time and effort required to build interactive AR tutorials by automatically generating key components of the AR tutorial while the author is assembling the physical pieces.
Further, the system guides authors through the process of adding videos, pictures, text and animations to the tutorial.
This concurrent assembly and tutorial generation approach allows for authoring of portable tutorials that fit the preferences of different end users.

assistant: text: We present a mixed media assembly tutorial authoring system that streamlines creation of videos, images, text and dynamic instructions in situ.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps

Paper content:
Modern neural network architectures use structured linear transformations, such as low-rank matrices, sparse matrices, permutations, and the Fourier transform, to improve inference speed and reduce memory usage compared to general linear maps.
However, choosing which of the myriad structured transformations to use (and its associated parameterization) is a laborious task that requires trading off speed, space, and accuracy.
We consider a different approach: we introduce a family of matrices called kaleidoscope matrices (K-matrices) that provably capture any structured matrix with near-optimal space (parameter) and time (arithmetic operation) complexity.
We empirically validate that K-matrices can be automatically learned within end-to-end pipelines to replace hand-crafted procedures, in order to improve model quality.
For example, replacing channel shuffles in ShuffleNet improves classification accuracy on ImageNet by up to 5%.
Learnable K-matrices can also simplify hand-engineered pipelines---we replace filter bank feature computation in speech data preprocessing with a kaleidoscope layer, resulting in only 0.4% loss in accuracy on the TIMIT speech recognition task.
K-matrices can also capture latent structure in models: for a challenging permuted image classification task, adding a K-matrix to a standard convolutional architecture can enable learning the latent permutation and improve accuracy by over 8 points.
We provide a practically efficient implementation of our approach, and use K-matrices in a Transformer network to attain 36% faster end-to-end inference speed on a language translation task.

assistant: text: We propose a differentiable family of "kaleidoscope matrices," prove that all structured matrices can be represented in this form, and use them to replace hand-crafted linear maps in deep learning models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Hope For The Best But Prepare For The Worst: Cautious Adaptation In RL Agents

Paper content:
We study the problem of safe adaptation: given a model trained on a variety of past experiences for some task, can this model learn to perform that task in a new situation while avoiding catastrophic failure?
This problem setting occurs frequently in real-world reinforcement learning scenarios such as a vehicle adapting to drive in a new city, or a robotic drone adapting a policy trained only in simulation.
While learning without catastrophic failures is exceptionally difficult, prior experience can allow us to learn models that make this much easier.
These models might not directly transfer to new settings, but can enable cautious adaptation that is substantially safer than na\"{i}ve adaptation as well as learning from scratch.
Building on this intuition, we propose risk-averse domain adaptation (RADA).
RADA works in two steps: it first trains probabilistic model-based RL agents in a population of source domains to gain experience and capture epistemic uncertainty about the environment dynamics.
Then, when dropped into a new environment, it employs a pessimistic exploration policy, selecting actions that have the best worst-case performance as forecasted by the probabilistic model.
We show that this simple maximin policy accelerates domain adaptation in a safety-critical driving environment with varying vehicle sizes.
We compare our approach against other approaches for adapting to new environments, including meta-reinforcement learning.

assistant: text: Adaptation of an RL agent in a target environment with unknown dynamics is fast and safe when we transfer prior experience in a variety of environments and then select risk-averse actions during adaptation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Eligibility traces provide a data-inspired alternative to backpropagation through time

Paper content:
Learning in recurrent neural networks (RNNs) is most often implemented by gradient descent using backpropagation through time (BPTT), but BPTT does not model accurately how the brain learns.
Instead, many experimental results on synaptic plasticity can be summarized as three-factor learning rules involving eligibility traces of the local neural activity and a third factor.
We present here eligibility propagation (e-prop), a new factorization of the loss gradients in RNNs that fits the framework of three factor learning rules when derived for biophysical spiking neuron models.
When tested on the TIMIT speech recognition benchmark, it is competitive with BPTT both for training artificial LSTM networks and spiking RNNs.
Further analysis suggests that the diversity of learning signals and the consideration of slow internal neural dynamics are decisive to the learning efficiency of e-prop.

assistant: text: We present eligibility propagation an alternative to BPTT that is compatible with experimental data on synaptic plasticity and competes with BPTT on machine learning benchmarks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Layerwise Learning Rates for Object Features in Unsupervised and Supervised Neural Networks And Consequent Predictions for the Infant Visual System

Paper content:
To understand how object vision develops in infancy and childhood, it will be necessary to develop testable computational models.
Deep neural networks (DNNs) have proven valuable as models of adult vision, but it is not yet clear if they have any value as models of development.
As a first model, we measured learning in a DNN designed to mimic the architecture and representational geometry of the visual system (CORnet).
We quantified the development of explicit object representations at each level of this network through training by freezing the convolutional layers and training an additional linear decoding layer.
We evaluate decoding accuracy on the whole ImageNet validation set, and also for individual visual classes.
CORnet, however, uses supervised training and because infants have only extremely impoverished access to labels they must instead learn in an unsupervised manner.
We therefore also measured learning in a state-of-the-art unsupervised network (DeepCluster).
CORnet and DeepCluster differ in both supervision and in the convolutional networks at their heart, thus to isolate the effect of supervision, we ran a control experiment in which we trained the convolutional network from DeepCluster (an AlexNet variant) in a supervised manner.
We make predictions on how learning should develop across brain regions in infants.
In all three networks, we also tested for a relationship in the order in which infants and machines acquire visual classes, and found only evidence for a counter-intuitive relationship.
We discuss the potential reasons for this.

assistant: text: Unsupervised networks learn from bottom up; machines and infants acquire visual classes in different orders
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning to solve the credit assignment problem

Paper content:
Backpropagation is driving today's artificial neural networks.
However, despite extensive research, it remains unclear if the brain implements this algorithm.
Among neuroscientists, reinforcement learning (RL) algorithms are often seen as a realistic alternative.
However, the convergence rate of such learning scales poorly with the number of involved neurons.
Here we propose a hybrid learning approach, in which each neuron uses an RL-type strategy to learn how to approximate the gradients that backpropagation would provide.
We show that our approach learns to approximate the gradient, and can match the performance of gradient-based learning on fully connected and convolutional networks.
Learning feedback weights provides a biologically plausible mechanism of achieving good performance, without the need for precise, pre-specified learning rules.

assistant: text: Perturbations can be used to learn feedback weights on large fully connected and convolutional networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Hierarchical Representations for Efficient Architecture Search

Paper content:
We explore efficient neural architecture search methods and show that a simple yet powerful evolutionary algorithm can discover new architectures with excellent performance.
Our approach combines a novel hierarchical genetic representation scheme that imitates the modularized design pattern commonly adopted by human experts, and an expressive search space that supports complex topologies.
Our algorithm efficiently discovers architectures that outperform a large number of manually designed models for image classification, obtaining top-1 error of 3.6% on CIFAR-10 and 20.3% when transferred to ImageNet, which is competitive with the best existing neural architecture search approaches.
We also present results using random search, achieving 0.3% less top-1 accuracy on CIFAR-10 and 0.1% less on ImageNet whilst reducing the search time from 36 hours down to 1 hour.

assistant: text: In this paper we propose a hierarchical architecture representation in which doing random or evolutionary architecture search yields highly competitive results using fewer computational resources than the prior art.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Deep Learning Approach for Survival Clustering without End-of-life Signals

Paper content:
The goal of survival clustering is to map subjects (e.g., users in a social network, patients in a medical study) to $K$ clusters ranging from low-risk to high-risk.
Existing survival methods assume the presence of clear \textit{end-of-life} signals or introduce them artificially using a pre-defined timeout.
In this paper, we forego this assumption and introduce a loss function that differentiates between the empirical lifetime distributions of the clusters using a modified Kuiper statistic.
We learn a deep neural network by optimizing this loss, that performs a soft clustering of users into survival groups.
We apply our method to a social network dataset with over 1M subjects, and show significant improvement in C-index compared to alternatives.

assistant: text: The goal of survival clustering is to map subjects into clusters. Without end-of-life signals, this is a challenging task. To address this task we propose a new loss function by modifying the Kuiper statistics.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Unsupervised Deep Basis Pursuit: Learning inverse problems without ground-truth data

Paper content:
Basis pursuit is a compressed sensing optimization in which the l1-norm is minimized subject to model error constraints.
Here we use a deep neural network prior instead of l1-regularization.
Using known noise statistics, we jointly learn the prior and reconstruct images without access to ground-truth data.
During training, we use alternating minimization across an unrolled iterative network and jointly solve for the neural network weights and training set image reconstructions.
At inference, we fix the weights and pass the measurements through the network.
We compare reconstruction performance between unsupervised and supervised (i.e. with ground-truth) methods.
We hypothesize this technique could be used to learn reconstruction when ground-truth data are unavailable, such as in high-resolution dynamic MRI.

assistant: text: We present an unsupervised deep learning reconstruction for imaging inverse problems that combines neural networks with model-based constraints.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Disagreement-Regularized Imitation Learning

Paper content:
We present a simple and effective algorithm designed to address the covariate shift problem in imitation learning.
It operates by training an ensemble of policies on the expert demonstration data, and using the variance of their predictions as a cost which is minimized with RL together with a supervised behavioral cloning cost.
Unlike adversarial imitation methods, it uses a fixed reward function which is easy to optimize.
We prove a regret bound for the algorithm in the tabular setting which is linear in the time horizon multiplied by a coefficient which we show to be low for certain problems in which behavioral cloning fails.
We evaluate our algorithm empirically across multiple pixel-based Atari environments and continuous control tasks, and show that it matches or significantly outperforms behavioral cloning and generative adversarial imitation learning.

assistant: text: Method for addressing covariate shift in imitation learning using ensemble uncertainty
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: MULTIPOLAR: Multi-Source Policy Aggregation for Transfer Reinforcement Learning between Diverse Environmental Dynamics

Paper content:
Transfer reinforcement learning (RL) aims at improving learning efficiency of an agent by exploiting knowledge from other source agents trained on relevant tasks.
However, it remains challenging to transfer knowledge between different environmental dynamics without having access to the source environments.
In this work, we explore a new challenge in transfer RL, where only a set of source policies collected under unknown diverse dynamics is available for learning a target task efficiently.
To address this problem, the proposed approach, MULTI-source POLicy AggRegation (MULTIPOLAR), comprises two key techniques.
We learn to aggregate the actions provided by the source policies adaptively to maximize the target task performance.
Meanwhile, we learn an auxiliary network that predicts residuals around the aggregated actions, which ensures the target policy's expressiveness even when some of the source policies perform poorly.
We demonstrated the effectiveness of MULTIPOLAR through an extensive experimental evaluation across six simulated environments ranging from classic control problems to challenging robotics simulations, under both continuous and discrete action spaces.

assistant: text: We propose MULTIPOLAR, a transfer RL method that leverages a set of source policies collected under unknown diverse environmental dynamics to efficiently learn a target policy in another dynamics.