ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Improving Sample Efficiency in Model-Free Reinforcement Learning from Images

Paper content:
Training an agent to solve control tasks directly from high-dimensional images with model-free reinforcement learning (RL) has proven difficult.
The agent needs to learn  a latent representation together with a control policy to perform the task.
Fitting a high-capacity encoder using a scarce reward signal is not only extremely sample inefficient, but also prone to suboptimal convergence.
Two ways to improve sample efficiency are to learn a good feature representation and use off-policy algorithms.
We dissect various approaches of learning good latent features, and conclude that the image reconstruction loss is the essential ingredient that enables efficient and stable representation learning in image-based RL.
Following these findings, we devise an off-policy actor-critic algorithm with an auxiliary decoder that trains end-to-end and matches state-of-the-art performance across both model-free and model-based algorithms on many challenging control tasks.
We release our code to encourage future research on image-based RL.

assistant: text: We design a simple and efficient model-free off-policy method for image-based reinforcement learning that matches the state-of-the-art model-based methods in sample efficiency
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep Connectomics Networks: Neural Network Architectures Inspired by Neuronal Networks

Paper content:
The interplay between inter-neuronal network topology and cognition has been studied deeply by connectomics researchers and network scientists, which is crucial towards understanding the remarkable efficacy of biological neural networks.
Curiously, the deep learning revolution that revived neural networks has not paid much attention to topological aspects.
The architectures of deep neural networks (DNNs) do not resemble their biological counterparts in the topological sense.
We bridge this gap by presenting initial results of Deep Connectomics Networks (DCNs) as DNNs with topologies inspired by real-world neuronal networks.
We show high classification accuracy obtained by DCNs whose architecture was inspired by the biological neuronal networks of C. Elegans and the mouse visual cortex.

assistant: text: Initial findings in the intersection of network neuroscience and deep learning. C. Elegans and a mouse visual cortex learn to recognize handwritten digits.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning to play slot cars and Atari 2600 games in just minutes

Paper content:
Machine learning algorithms for controlling devices will need to learn quickly, with few trials.
Such a goal can be attained with concepts borrowed from continental philosophy and formalized using tools from the mathematical theory of categories.
Illustrations of this approach are presented on a cyberphysical system: the slot car game, and also on Atari 2600 games.
There is a growing need for algorithms that control cyberphysical systems to learn with very little data how to operate quickly in a partially-known environment.
Many reinforcement-learning (RL) solutions using neural networks (NN) have proved to work well with emulators, for instance with the Atari 1 2600 games BID17 , or with real systems such as robots BID11 .
However, these state-of-the-art approaches need a lot of training data, which may not be obtainable within the allowed time frame or budget.
This work thus started as an alternative approach to teach computers to learn quickly to perform as efficiently as the existing solution with approximately one percent of the training data, time, and computing resources.We first review reinforcement learning methods for Markov Decision Processes (MDP) and Partially Observable MDP (POMDP).
We then explain the motivation behind our continental-philosophyinspired approach.
We describe the two classes of problems on which we focus: the bijective case, which may lead to playing by imitating, and the category-based approach, which should lead to a more innovative behavior of the control algorithms.
Both approaches rely on knowledge accumulated during previous experiences, as in Lifelong Machine Learning BID6 .These
two approaches are illustrated by results from both a commercial slot car game controlled by an 8-bit Arduino system, and from Atari 2600 video games running within the Arcade Learning Environment (ALE, see BID1 ).
Continental philosophy lead us to formalize a mathematical concept to control an agent evolving in a world, whether it is simulated or real.
The power of this framework was illustrated by the theoretical example of the slot car on unknown circuits.
Results from experiments with a real slot car, using real analog signals confirmed our expectations, even though it only used a basic survival approach.
Moreover, the same basic survival strategy was applied to two Atari 2600 games and showed the same trend: even though not as skilled as, for instance, DQN-based agents trained with two hundred million frames, our AI reached in less than ten thousand frames scores that DQN met after learning with a few million frames.The next steps are to apply the transposition properties to the Atari games, as we did for the slot car, which should further decrease the learning time when playing a new game.
Moreover, going beyond the basic survival strategy will be mandatory to reach higher scores: approaches based on Monte-Carlo Tree Search will be investigated.

assistant: text: Continental-philosophy-inspired approach to learn with few data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Fooling Detection Alone is Not Enough: Adversarial Attack against Multiple Object Tracking

Paper content:
Recent work in adversarial machine learning started to focus on the visual perception in autonomous driving and studied Adversarial Examples (AEs) for object detection models.
However, in such visual perception pipeline the detected objects must also be tracked, in a process called Multiple Object Tracking (MOT), to build the moving trajectories of surrounding obstacles.
Since MOT is designed to be robust against errors in object detection, it poses a general challenge to existing attack techniques that blindly target objection detection: we find that a success rate of over 98% is needed for them to actually affect the tracking results, a requirement that no existing attack technique can satisfy.
In this paper, we are the first to study adversarial machine learning attacks against the complete visual perception pipeline in autonomous driving, and discover a novel attack technique, tracker hijacking, that can effectively fool MOT using AEs on object detection.
Using our technique, successful AEs on as few as one single frame can move an existing object in to or out of the headway of an autonomous vehicle to cause potential safety hazards.
We perform evaluation using the Berkeley Deep Drive dataset and find that on average when 3 frames are attacked, our attack can have a nearly 100% success rate while attacks that blindly target object detection only have up to 25%.

assistant: text: We study the adversarial machine learning attacks against the Multiple Object Tracking mechanisms for the first time.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Interpreting Word Embeddings with Eigenvector Analysis

Paper content:
Dense word vectors have proven their values in many downstream NLP tasks over the past few years.
However, the dimensions of such embeddings are not easily interpretable.
Out of the d-dimensions in a word vector, we would not be able to understand what high or low values mean.
Previous approaches addressing this issue have mainly focused on either training sparse/non-negative constrained word embeddings, or post-processing standard pre-trained word embeddings.
On the other hand, we analyze conventional word embeddings trained with Singular Value Decomposition, and reveal similar interpretability.
We use a novel eigenvector analysis method inspired from Random Matrix Theory and show that semantically coherent groups not only form in the row space, but also the column space.
This allows us to view individual word vector dimensions as human-interpretable semantic features.

assistant: text: Without requiring any constraints or post-processing, we show that the salient dimensions of word vectors can be interpreted as semantic features.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Wasserstein Embeddings

Paper content:
The Wasserstein distance received a lot of attention recently in the community of machine learning, especially for its principled way of comparing distributions.
It has found numerous applications in several hard problems, such as domain adaptation, dimensionality reduction or generative models.
However, its use is still limited by a heavy computational cost.
Our goal is to alleviate this problem by providing an approximation mechanism that allows to break its inherent complexity.
It relies on the search of an embedding where the Euclidean distance mimics the Wasserstein distance.
We show that such an embedding can be found with a siamese architecture associated with a decoder network that allows to move from the embedding space back to the original input space.
Once this embedding has been found, computing optimization problems in the Wasserstein space (e.g. barycenters, principal directions or even archetypes) can be conducted extremely fast.
Numerical experiments supporting this idea are conducted on image datasets, and show the wide potential benefits of our method.

assistant: text: We show that it is possible to fastly approximate Wasserstein distances computation by finding an appropriate embedding where Euclidean distance emulates the Wasserstein distance
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: SoftAdam: Unifying SGD and Adam for better stochastic gradient descent

Paper content:
Abstract Stochastic gradient descent (SGD) and Adam are commonly used to optimize deep neural networks, but choosing one usually means making tradeoffs between speed, accuracy and stability.
Here we present an intuition for why the tradeoffs exist as well as a method for unifying the two in a continuous way.
This makes it possible to control the way models are trained in much greater detail.
We show that for default parameters, the new algorithm equals or outperforms SGD and Adam across a range of models for image classification tasks and outperforms SGD for language modeling tasks.

assistant: text: An algorithm for unifying SGD and Adam and empirical study of its performance
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: word2ket: Space-efficient Word Embeddings inspired by Quantum Entanglement

Paper content:
Deep learning natural language processing models often use vector word embeddings, such as word2vec or GloVe, to represent words.
A discrete sequence of words can be much more easily integrated with downstream neural layers if it is represented as a  sequence of continuous vectors.
Also, semantic relationships between words, learned from a text corpus, can be encoded in the relative configurations of the embedding vectors.
However, storing and accessing embedding vectors for all words in a dictionary requires large amount of space, and may stain systems with limited GPU memory.
Here, we used approaches inspired by quantum computing to propose two related methods, word2ket and word2ketXS, for storing word embedding matrix during training and inference in a highly efficient way.
Our approach achieves a hundred-fold or more reduction in the space required to store the embeddings with almost no relative drop in accuracy in practical natural language processing tasks.

assistant: text: We use ideas from quantum computing to proposed word embeddings that utilize much fewer trainable parameters.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Comparative Study of Lexical and Semantic Emoji Suggestion Systems

Paper content:
Emoji suggestion systems based on typed text have been proposed to encourage emoji usage and enrich text messaging; however, such systems’ actual effects on the chat experience remain unknown.
We built an Android keyboard with both lexical (word-based) and semantic (meaning-based) emoji suggestion capabilities and compared these in two different studies.
To investigate the effect of emoji suggestion in online conversations, we conducted a laboratory text-messaging study with 24 participants, and also a 15-day longitudinal field deployment with 18 participants.
We found that lexical emoji suggestions increased emoji usage by 31.5% over a keyboard without suggestions, while semantic suggestions increased emoji usage by 125.1%.
However, suggestion mechanisms did not affect the chatting experience significantly.
From these studies, we formulate a set of design guidelines for future emoji suggestion systems that better support users’ needs.

assistant: text: We built an Android keyboard with both lexical (word-based) and semantic (meaning-based) emoji suggestion capabilities and compared their effects in two different chat studies.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Bayesian Meta Sampling for Fast Uncertainty Adaptation

Paper content:
Meta learning has been making impressive progress for fast model adaptation.
However, limited work has been done on learning fast uncertainty adaption for Bayesian modeling.
In this paper, we propose to achieve the goal by placing meta learning on the space of probability measures, inducing the concept of meta sampling for fast uncertainty adaption.
Specifically, we propose a Bayesian meta sampling framework consisting of two main components: a meta sampler and a sample adapter.
The meta sampler is constructed by adopting a neural-inverse-autoregressive-flow (NIAF) structure, a variant of the recently proposed neural autoregressive flows, to efficiently generate meta samples to be adapted.
The sample adapter moves meta samples to task-specific samples, based on a newly proposed and general Bayesian sampling technique, called optimal-transport Bayesian sampling.
The combination of the two components allows a simple learning procedure for the
meta sampler to be developed, which can be efficiently optimized via standard back-propagation.
Extensive experimental results demonstrate the efficiency and effectiveness of the proposed framework, obtaining better sample quality and faster
uncertainty adaption compared to related methods.

assistant: text: We proposed a Bayesian meta sampling method for adapting the model uncertainty in meta learning
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Causal Reasoning from Meta-reinforcement learning

Paper content:
Discovering and exploiting the causal structure in the environment is a crucial challenge for intelligent agents.
Here we explore whether modern deep reinforcement learning can be used to train agents to perform causal reasoning.
We adopt a meta-learning approach, where the agent learns a policy for conducting experiments via causal interventions, in order to support a subsequent task which rewards making accurate causal inferences.We also found the agent could make sophisticated counterfactual predictions, as well as learn to draw causal inferences from purely observational data.
Though powerful formalisms for causal reasoning have been developed, applying them in real-world domains can be difficult because fitting to large amounts of high dimensional data often requires making idealized assumptions.
Our results suggest that causal reasoning in complex settings may benefit from powerful learning-based approaches.
More generally, this work may offer new strategies for structured exploration in reinforcement learning, by providing agents with the ability to perform—and interpret—experiments.

assistant: text: meta-learn a learning algorithm capable of causal reasoning
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Graph Classification with Geometric Scattering

Paper content:
One of the most notable contributions of deep learning is the application of convolutional neural networks (ConvNets) to structured signal classification, and in particular image classification.
Beyond their impressive performances in supervised learning, the structure of such networks inspired the development of deep filter banks referred to as scattering transforms.
These transforms apply a cascade of wavelet transforms and complex modulus operators to extract features that are invariant to group operations and stable to deformations.
Furthermore, ConvNets inspired recent advances in geometric deep learning, which aim to generalize these networks to graph data by applying notions from graph signal processing to learn deep graph filter cascades.
We further advance these lines of research by proposing a geometric scattering transform using graph wavelets defined in terms of random walks on the graph.
We demonstrate the utility of features extracted with this designed deep filter bank in graph classification of biochemistry and social network data (incl. state of the art results in the latter case), and in data exploration, where they enable inference of EC exchange preferences in enzyme evolution.

assistant: text: We present a new feed forward graph ConvNet based on generalizing the wavelet scattering transform of Mallat, and demonstrate its utility in graph classification and data exploration tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Weighted Transformer Network for Machine Translation

Paper content:
State-of-the-art results on neural machine translation often use attentional sequence-to-sequence models with some form of convolution or recursion.
Vaswani et.
al. (2017) propose a new architecture that avoids recurrence and convolution completely.
Instead, it uses only self-attention and feed-forward layers.
While the proposed architecture achieves state-of-the-art results on several machine translation tasks, it requires a large number of parameters and training iterations to converge.
We propose Weighted Transformer, a Transformer with modified attention layers, that not only outperforms the baseline network in BLEU score but also converges 15-40% faster.
Specifically, we replace the multi-head attention by multiple self-attention branches that the model learns to combine during the training process.
Our model improves the state-of-the-art performance by 0.5 BLEU points on the WMT 2014 English-to-German translation task and by 0.4 on the English-to-French translation task.

assistant: text: Using branched attention with learned combination weights outperforms the baseline transformer for machine translation tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Rethinking Curriculum Learning With Incremental Labels And Adaptive Compensation

Paper content:
Like humans, deep networks learn better when samples are organized and introduced in a meaningful order or curriculum.
While conventional approaches to curriculum learning emphasize the difficulty of samples as the core incremental strategy, it forces networks to learn from small subsets of data while introducing pre-computation overheads.
In this work, we propose Learning with Incremental Labels and Adaptive Compensation (LILAC), which introduces a novel approach to curriculum learning.
LILAC emphasizes incrementally learning labels instead of incrementally learning difficult samples.
It works in two distinct phases: first, in the incremental label introduction phase, we unmask ground-truth labels in fixed increments during training, to improve the starting point from which networks learn.
In the adaptive compensation phase, we compensate for failed predictions by adaptively altering the target vector to a smoother distribution.
We evaluate LILAC against the closest comparable methods in batch and curriculum learning and label smoothing, across three standard image benchmarks, CIFAR-10, CIFAR-100, and STL-10.
We show that our method outperforms batch learning with higher mean recognition accuracy as well as lower standard deviation in performance consistently across all benchmarks.
We further extend LILAC to state-of-the-art performance across CIFAR-10 using simple data augmentation while exhibiting label order invariance among other important properties.

assistant: text: A novel approach to curriculum learning by incrementally learning labels and adaptively smoothing labels for mis-classified samples which boost average performance and decreases standard deviation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Understanding Deep Neural Networks with Rectified Linear Units

Paper content:
In this paper we investigate the family of functions representable by deep neural networks (DNN) with rectified linear units (ReLU).
We give an algorithm to train a ReLU DNN with one hidden layer to {\em global optimality} with runtime polynomial in the data size albeit exponential in the input dimension.
Further, we improve on the known lower bounds on size (from exponential to super exponential) for approximating a ReLU deep net function by a shallower ReLU net.
Our gap theorems hold for smoothly parametrized families of ``hard'' functions, contrary to countable, discrete families known in the literature.
An example consequence of our gap theorems is the following: for every natural number $k$ there exists a function representable by a ReLU DNN with $k^2$ hidden layers and total size $k^3$, such that any ReLU DNN with at most $k$ hidden layers will require at least $\frac12k^{k+1}-1$ total nodes.
Finally, for the family of $\R^n\to \R$ DNNs with ReLU activations, we show a new lowerbound on the number of affine pieces, which is larger than previous constructions in certain regimes of the network architecture and most distinctively our lowerbound is demonstrated by an explicit construction of a \emph{smoothly parameterized} family of functions attaining this scaling.
Our construction utilizes the theory of zonotopes from polyhedral theory.

assistant: text: This paper 1) characterizes functions representable by ReLU DNNs, 2) formally studies the benefit of depth in such architectures,  3) gives an algorithm to implement empirical risk minimization to global optimality for two layer ReLU nets.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: An Etching Latte Art Support System by Tracing the Making Procedure Based on Projection Mapping

Paper content:
It is difficult for the beginners of etching latte art to make well-balanced patterns by using two fluids with different viscosities such as foamed milk and syrup.
Even though making etching latte art while watching making videos which show the procedure, it is difficult to keep balance.
Thus well-balanced etching latte art cannot be made easily.
In this paper, we propose a system which supports the beginners to make well-balanced etching latte art by projecting a making procedure of etching latte art directly onto a cappuccino.
The experiment results show the progress by using our system.
We also discuss about the similarity of the etching latte art and the design templates by using background subtraction.

assistant: text: We have developed an etching latte art support system which projects the making procedure directly onto a cappuccino to help the beginners to make well-balanced etching latte art.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Which Tasks Should Be Learned Together in Multi-task Learning?

Paper content:
Many computer vision applications require solving multiple tasks in real-time.
A neural network can be trained to solve multiple tasks simultaneously using 'multi-task learning'.
This saves computation at inference time as only a single network needs to be evaluated.
Unfortunately, this often leads to inferior overall performance as task objectives compete, which consequently poses the question: which tasks should and should not be learned together in one network when employing multi-task learning?
We systematically study task cooperation and competition and propose a framework for assigning tasks to a few neural networks such that cooperating tasks are computed by the same neural network, while competing tasks are computed by different networks.
Our framework offers a time-accuracy trade-off and can produce better accuracy using less inference time than not only a single large multi-task neural network but also many single-task networks.

assistant: text: We analyze what tasks are best learned together in one network, and which are best to learn separately.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models

Paper content:
Deep neural networks have achieved impressive performance in handling complicated semantics in natural language, while mostly treated as black boxes.
To explain how the model handles compositional semantics of words and phrases, we study the hierarchical explanation problem.
We highlight the key challenge is to compute non-additive and context-independent importance for individual words and phrases.
We show some prior efforts on hierarchical explanations, e.g. contextual decomposition,  do not satisfy the desired properties mathematically, leading to inconsistent explanation quality in different models.
In this paper, we propose a formal way to quantify the importance of each word or phrase to generate hierarchical explanations.
We modify contextual decomposition algorithms according to our formulation, and propose a model-agnostic explanation algorithm with competitive performance.
Human evaluation and automatic metrics evaluation on both LSTM models and fine-tuned BERT Transformer models on multiple datasets show that our algorithms robustly outperform prior works on hierarchical explanations.
We show our algorithms help explain compositionality of semantics, extract classification rules, and improve human trust of models.

assistant: text: We propose measurement of phrase importance and algorithms for hierarchical explanation of neural sequence model predictions
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Tensor Analysis on Dense Connectivity via Convolutional Arithmetic Circuits

Paper content:
Several state of the art convolutional networks rely on inter-connecting different layers to ease the flow of information and gradient between their input and output layers.
These techniques have enabled practitioners to successfully train deep convolutional networks with hundreds of layers.
Particularly, a novel way of interconnecting layers was introduced as the Dense Convolutional Network (DenseNet) and has achieved state of the art performance on relevant image recognition tasks.
Despite their notable empirical success, their theoretical understanding is still limited.
In this work, we address this problem by analyzing the effect of layer interconnection on the overall expressive power of a convolutional network.
In particular, the  connections used in DenseNet are compared with other types of inter-layer connectivity.
We carry out a tensor analysis on the expressive power inter-connections on convolutional arithmetic circuits (ConvACs) and relate our results to standard convolutional networks.
The analysis leads to performance bounds and practical guidelines for design of ConvACs.
The generalization of these results are discussed for other kinds of convolutional networks via generalized tensor decompositions.

assistant: text: We analyze the expressive power of the connections used in DenseNets via tensor decompositions.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Guided variational autoencoder for disentanglement learning

Paper content:
We propose an algorithm, guided variational autoencoder (Guided-VAE), that is able to learn a controllable generative model by performing latent representation disentanglement learning.
The learning objective is achieved by providing signal to the latent encoding/embedding in VAE without changing its main backbone architecture, hence retaining the desirable properties of the VAE.
We design an unsupervised and a supervised strategy in Guided-VAE and observe enhanced modeling and controlling capability over the vanilla VAE.
In the unsupervised strategy, we guide the VAE learning by introducing a lightweight decoder that learns latent geometric transformation and principal components; in the supervised strategy, we use an adversarial excitation and inhibition mechanism to encourage the disentanglement of the latent variables.
Guided-VAE enjoys its transparency and simplicity for the general representation learning task, as well as disentanglement learning.
On a number of experiments for representation learning, improved synthesis/sampling, better disentanglement for classification, and reduced classification errors in meta learning have been observed.

assistant: text: Learning a controllable generative model by performing latent representation disentanglement learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Prior Convictions: Black-box Adversarial Attacks with Bandits and Priors

Paper content:
We study the problem of generating adversarial examples in a black-box setting in which only loss-oracle access to a model is available.
We introduce a framework that conceptually unifies much of the existing work on black-box attacks, and demonstrate that the current state-of-the-art methods are optimal in a natural sense.
Despite this optimality, we show how to improve black-box attacks by bringing a new element into the problem: gradient priors.
We give a bandit optimization-based algorithm that allows us to seamlessly integrate any such priors, and we explicitly identify and incorporate two examples.
The resulting methods use two to four times fewer queries and fail two to five times less than the current state-of-the-art.
The code for reproducing our work is available at https://git.io/fAjOJ.

assistant: text: We present a unifying view on black-box adversarial attacks as a gradient estimation problem, and then present a framework (based on bandits optimization) to integrate priors into gradient estimation, leading to significantly increased performance.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Fast Node Embeddings: Learning Ego-Centric Representations

Paper content:
Representation learning is one of the foundations of Deep Learning and allowed important improvements on several Machine Learning tasks, such as Neural Machine Translation, Question Answering and Speech Recognition.
Recent works have proposed new methods for learning representations for nodes and edges in graphs.
Several of these methods are based on the SkipGram algorithm, and they usually process a large number of multi-hop neighbors in order to produce the context from which node representations are learned.
In this paper, we propose an effective and also efficient method for generating node embeddings in graphs that employs a restricted number of permutations over the immediate neighborhood of a node as context to generate its representation, thus ego-centric representations.
We present a thorough evaluation showing that our method outperforms state-of-the-art methods in six different datasets related to the problems of link prediction and node classification, being one to three orders of magnitude faster than baselines when generating node embeddings for very large graphs.

assistant: text: A faster method for generating node embeddings that employs a number of permutations over a node's immediate neighborhood as context to generate its representation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: PAC-Bayes Few-shot Meta-learning with Implicit Learning of Model Prior Distribution

Paper content:
We introduce a new and rigorously-formulated PAC-Bayes few-shot meta-learning algorithm that implicitly learns a model prior distribution of interest.
Our proposed method extends the PAC-Bayes framework from a single task setting to the few-shot meta-learning setting to upper-bound generalisation errors on unseen tasks.
We  also propose a generative-based approach to model the shared prior and task-specific posterior more expressively compared to the usual diagonal Gaussian assumption.
We show that the models trained with our proposed meta-learning algorithm are well calibrated and accurate, with state-of-the-art calibration and classification results on mini-ImageNet benchmark, and competitive results in a multi-modal task-distribution regression.

assistant: text: Bayesian meta-learning using PAC-Bayes framework and implicit prior distributions
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Answering Science Exam Questions Using Query Reformulation with Background Knowledge

Paper content:
Open-domain question answering (QA) is an important problem in AI and NLP that is emerging as a bellwether for progress on the generalizability of AI methods and techniques.
Much of the progress in open-domain QA systems has been realized through advances in information retrieval methods and corpus construction.
In this paper, we focus on the recently introduced ARC Challenge dataset, which contains 2,590 multiple choice questions authored for grade-school science exams.
These questions are selected to be the most challenging for current QA systems, and current state of the art performance is only slightly better than random chance.
We present a system that reformulates a given question into queries that are used to retrieve supporting text from a large corpus of science-related text.
Our rewriter is able to incorporate background knowledge from ConceptNet and -- in tandem with a generic textual entailment system trained on SciTail that identifies support in the retrieved results -- outperforms several strong baselines on the end-to-end QA task despite only being trained to identify essential terms in the original source question.
We use a generalizable decision methodology over the retrieved evidence and answer candidates to select the best answer.
By combining query reformulation, background knowledge, and textual entailment our system is able to outperform several strong baselines on the ARC dataset.

assistant: text: We explore how using background knowledge with query reformulation can help retrieve better supporting evidence when answering multiple-choice science questions.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Language Style Transfer from Non-Parallel Text with Arbitrary Styles

Paper content:
Language style transfer is the problem of migrating the content of a source sentence to a target style.
In many applications, parallel training data are not available and source sentences to be transferred may have arbitrary and unknown styles.
In this paper, we present an encoder-decoder framework under this problem setting.
Each sentence is encoded into its content and style latent representations.
By recombining the content with the target style, we can decode a sentence aligned in the target domain.
To adequately constrain the encoding and decoding functions, we couple them with two loss functions.
The first is a style discrepancy loss, enforcing that the style representation accurately encodes the style information guided by the discrepancy between the sentence style and the target style.
The second is a cycle consistency loss, which ensures that the transferred sentence should preserve the content of the original sentence disentangled from its style.
We validate the effectiveness of our proposed model on two tasks: sentiment modification of restaurant reviews, and dialog response revision with a romantic style.

assistant: text: We present an encoder-decoder framework for language style transfer, which allows for the use of non-parallel data and source data with various unknown language styles.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: IB-GAN: Disentangled Representation Learning with Information Bottleneck GAN

Paper content:
We present a novel architecture of GAN for a disentangled representation learning.
The new model architecture is inspired by Information Bottleneck (IB) theory thereby named IB-GAN.
IB-GAN objective is similar to that of InfoGAN but has a crucial difference; a capacity regularization for mutual information is adopted, thanks to which the generator of IB-GAN can harness a latent representation in disentangled and interpretable manner.
To facilitate the optimization of IB-GAN in practice, a new variational upper-bound is derived.
With experiments on CelebA, 3DChairs, and dSprites datasets, we demonstrate that the visual quality of samples generated by IB-GAN is often better than those by β-VAEs.
Moreover, IB-GAN achieves much higher disentanglement metrics score than β-VAEs or InfoGAN on the dSprites dataset.

assistant: text: Inspired by Information Bottleneck theory,  we propose a new architecture of GAN for a disentangled representation learning
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Line-Storm Ludic System: An Interactive Augmented Stylus and Writing Pad for Creative Soundscape

Paper content:
We present Line-Storm, an interactive computer system for creative performance.
The context we investigated was writing on paper using Line-Storm.
We used self-report questionnaires as part of research involving human participants, to evaluate Line-Storm.
Line-Storm consisted of a writing stylus and writing pad, augmented with electronics.
The writing pad was connected to a contact microphone, and the writing stylus had a small micro-controller board and peripherals attached to it.
The signals from these electronic augmentations were fed into the audio-synthesis environment Max/MSP to produce an interactive soundscape.
We attempted to discover whether Line-Storm enhanced a self-reported sense of being present and engaged during a writing task, and we compared Line-Storm to a non-interactive control condition.
After performing statistical analysis in SPSS, we were unable to support our research hypothesis, that presence and engagement were enhanced by Line-Storm.
Participants reported they were, on average, no more present and engaged during the experimental condition than during the control condition.
As creativity is subtle, and varies with person, time, context,  space and so many other factors, this result was somewhat expected by us.
A statistically significant result of our study is that some participants responded to Line-Storm more positively than others.
These Preservers of Line-Storm were a group, distinct from other participants, who reported greater presence and engagement and who wrote more words with Line-Storm and during the control condition.
We discuss the results of our research and place Line-Storm in an artistic-technological context, drawing upon writings by Martin Heidegger when considering the nature of Line-Storm.
Future work includes modifying interactive components, improving aesthetics and using more miniaturized electronics, experimenting with a drawing task instead of a writing task, and collaborating with a composer of electronic music to make a more interesting, immersive, and engaging interactive soundscape for writing or drawing performance.

assistant: text: Interactive stylus based sound incorporating writing system
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Capacity of Deep Neural Networks under Parameter Quantization

Paper content:
Most deep neural networks (DNNs) require complex models to achieve high performance.
Parameter quantization is widely used for reducing the implementation complexities.
Previous studies on quantization were mostly based on extensive simulation using training data.
We choose a different approach and attempt to measure the per-parameter capacity of DNN models and interpret the results to obtain insights on optimum quantization of parameters.
This research uses artificially generated data and generic forms of fully connected DNNs, convolutional neural networks, and recurrent neural networks.
We conduct memorization and classification tests to study the effects of the number and precision of the parameters on the performance.
The model and the per-parameter capacities are assessed by measuring the mutual information between the input and the classified output.
We also extend the memorization capacity measurement results to image classification and language modeling tasks.
To get insight for parameter quantization when performing real tasks, the training and test performances are compared.

assistant: text: We suggest the sufficient number of bits for representing weights of DNNs and the optimum bits are conservative when solving real problems.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Side-Tuning: Network Adaptation via Additive Side Networks

Paper content:
When training a neural network for a desired task, one may prefer to adapt a pretrained network rather than start with a randomly initialized one -- due to lacking enough training data, performing lifelong learning where the system has to learn a new task while being previously trained for other tasks, or wishing to encode priors in the network via preset weights.
The most commonly employed approaches for network adaptation are fine-tuning and using the pre-trained network as a fixed feature extractor, among others.
In this paper we propose a straightforward alternative: Side-Tuning.
Side-tuning adapts a pretrained network by training a lightweight "side" network that is fused with the (unchanged) pre-rained network using a simple additive process.
This simple method works as well as or better than existing solutions while it resolves some of the basic issues with fine-tuning, fixed features, and several other common baselines.
In particular, side-tuning is less prone to overfitting when little training data is available, yields better results than using a fixed feature extractor, and doesn't suffer from catastrophic forgetting in lifelong learning.
We demonstrate the performance of side-tuning under a diverse set of scenarios, including lifelong learning (iCIFAR, Taskonomy), reinforcement learning, imitation learning (visual navigation in Habitat), NLP question-answering (SQuAD v2), and single-task transfer learning (Taskonomy), with consistently promising results.

assistant: text: Side-tuning adapts a pre-trained network by training a lightweight "side" network that is fused with the (unchanged) pre-trained network using a simple additive process.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Continual adaptation for efficient machine communication

Paper content:
To communicate with new partners in new contexts, humans rapidly form new linguistic conventions.
Recent language models trained with deep neural networks are able to comprehend and produce the existing conventions present in their training data, but are not able to flexibly and interactively adapt those conventions on the fly as humans do.
We introduce a repeated reference task as a benchmark for models of adaptation in communication and propose a regularized continual learning framework that allows an artificial agent initialized with a generic language model to more accurately and efficiently understand their partner over time.
We evaluate this framework through simulations on COCO and in real-time reference game experiments with human partners.

assistant: text: We propose a repeated reference benchmark task and a regularized continual learning approach for adaptive communication with humans in unfamiliar domains
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Correctness Verification of Neural Network

Paper content:
We present the first verification that a neural network for perception tasks produces
a correct output within a specified tolerance for every input of interest.
We define
correctness relative to a specification which identifies 1) a state space consisting of
all relevant states of the world and 2) an observation process that produces neural
network inputs from the states of the world.
Tiling the state and input spaces with
a finite number of tiles, obtaining ground truth bounds from the state tiles and
network output bounds from the input tiles, then comparing the ground truth and
network output bounds delivers an upper bound on the network output error for
any input of interest.
Results from two case studies highlight the ability of our
technique to deliver tight error bounds for all inputs of interest and show how the
error bounds vary over the state and input spaces.

assistant: text: We present the first verification that a neural network for perception tasks produces a correct output within a specified tolerance for every input of interest.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: RNNs with Private and Shared Representations for Semi-Supervised Sequence Learning

Paper content:
Training recurrent neural networks (RNNs) on long sequences using backpropagation through time (BPTT) remains a fundamental challenge.
It has been shown that adding a local unsupervised loss term into the optimization objective makes the training of RNNs on long sequences more effective.
While the importance of an unsupervised task can in principle be controlled by a coefficient in the objective function, the gradients with respect to the unsupervised loss term still influence all the hidden state dimensions, which might cause important information about the supervised task to be degraded or erased.
Compared to existing semi-supervised sequence learning methods, this paper focuses upon a traditionally overlooked mechanism -- an architecture with explicitly designed private and shared hidden units designed to mitigate the detrimental influence of the auxiliary unsupervised loss over the main supervised task.
We achieve this by dividing RNN hidden space into a private space for the supervised task and a shared space for both the supervised and unsupervised tasks.
We present extensive experiments with the proposed framework on several long sequence modeling benchmark datasets.
Results indicate that the proposed framework can yield performance gains in RNN models where long term dependencies are notoriously challenging to deal with.

assistant: text: This paper focuses upon a traditionally overlooked mechanism -- an architecture with explicitly designed private and shared hidden units designed to mitigate the detrimental influence of the auxiliary unsupervised loss over the main supervised task.