ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning To Avoid Negative Transfer in Few Shot Transfer Learning

Paper content:
Many tasks in natural language understanding require learning relationships between two sequences for various tasks such as natural language inference, paraphrasing and entailment.
These aforementioned tasks are similar in nature, yet they are often modeled individually.
Knowledge transfer can be effective for closely related tasks, which is usually carried out using parameter transfer in neural networks.
However, transferring all parameters, some of which irrelevant for a target task, can lead to sub-optimal results and can have a negative effect on performance, referred to as \textit{negative} transfer.
Hence, this paper focuses on the transferability of both instances and parameters across natural language understanding tasks by proposing an ensemble-based transfer learning method in the context of few-shot learning.
Our main contribution is a method for mitigating negative transfer across tasks when using neural networks, which involves dynamically bagging small recurrent neural networks trained on different subsets of the source task/s.
We present a straightforward yet novel approach for incorporating these networks to a target task for few-shot learning by using a decaying parameter chosen according to the slope changes of a smoothed spline error curve at sub-intervals during training.
Our proposed method show improvements over hard and soft parameter sharing transfer methods in the few-shot learning case and shows competitive performance against models that are trained given full supervision on the target task, from only few examples.

assistant: text: A dynamic bagging methods approach to avoiding negatve transfer in neural network few-shot transfer learning
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: GenDICE: Generalized Offline Estimation of Stationary Values

Paper content:
An important problem that arises in reinforcement learning and Monte Carlo methods is estimating quantities defined by the stationary distribution of a Markov chain.
In many real-world applications, access to the underlying transition operator is limited to a fixed set of data that has already been collected, without additional interaction with the environment being available.
We show that consistent estimation remains possible in this scenario, and that effective estimation can still be achieved in important applications.
Our approach is based on estimating a ratio that corrects for the discrepancy between the stationary and empirical distributions, derived from fundamental properties of the stationary distribution, and exploiting constraint reformulations based on variational divergence minimization.
The resulting algorithm, GenDICE, is straightforward and effective.
We prove the consistency of the method under general conditions, provide a detailed error analysis, and demonstrate strong empirical performance on benchmark tasks, including off-line PageRank and off-policy policy evaluation.

assistant: text: In this paper, we proposed a novel algorithm, GenDICE, for general stationary distribution correction estimation, which can handle both discounted and average off-policy evaluation on multiple behavior-agnostic samples.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Online Meta-Critic Learning for Off-Policy Actor-Critic Methods

Paper content:
Off-Policy Actor-Critic (Off-PAC) methods have proven successful in a variety of continuous control tasks.
Normally, the critic’s action-value function is updated using temporal-difference, and the critic in turn provides a loss for the actor that trains it to take actions with higher expected return.
In this paper, we introduce a novel and flexible meta-critic that observes the learning process and meta-learns an additional loss for the actor that accelerates and improves actor-critic learning.
Compared to the vanilla critic, the meta-critic network is explicitly trained to accelerate the learning process; and compared to existing meta-learning algorithms, meta-critic is rapidly learned online for a single task, rather than slowly over a family of tasks.
Crucially, our meta-critic framework is designed for off-policy based learners, which currently provide state-of-the-art reinforcement learning sample efficiency.
We demonstrate that online meta-critic learning leads to improvements in a variety of continuous control environments when combined with contemporary Off-PAC methods DDPG, TD3 and the state-of-the-art SAC.

assistant: text: We present Meta-Critic, an auxiliary critic module for off-policy actor-critic methods that can be meta-learned online during single task learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Demystifying MMD GANs

Paper content:
We investigate the training and performance of generative adversarial networks using the Maximum Mean Discrepancy (MMD) as critic, termed MMD GANs.
As our main theoretical contribution, we clarify the situation with bias in GAN loss functions raised by recent work: we show that gradient estimators used in the optimization process for both MMD GANs and Wasserstein GANs are unbiased, but learning a discriminator based on samples leads to biased gradients for the generator parameters.
We also discuss the issue of kernel choice for the MMD critic, and characterize the kernel corresponding to the energy distance used for the Cramér GAN critic.
Being an integral probability metric, the MMD benefits from training strategies recently developed for Wasserstein GANs.
In experiments, the MMD GAN is able to employ a smaller critic network than the Wasserstein GAN, resulting in a simpler and faster-training algorithm with matching performance.
We also propose an improved measure of GAN convergence, the Kernel Inception Distance, and show how to use it to dynamically adapt learning rates during GAN training.

assistant: text: Explain bias situation with MMD GANs; MMD GANs work with smaller critic networks than WGAN-GPs; new GAN evaluation metric.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: RNNSecureNet: Recurrent neural networks for Cybersecurity use-cases

Paper content:
Recurrent neural network(RNN) is an effective neural network in solving very complex supervised and unsupervised tasks.There has been a significant improvement in RNN field such as natural language processing, speech processing, computer vision and other multiple domains.
This paper deals with RNN application on different use cases like Incident Detection , Fraud Detection , and Android Malware Classification.
The best performing neural network architecture is chosen
by conducting different chain of experiments for different network parameters and structures.The network is run up to 1000 epochs with learning rate set in the range of 0.01 to 0.5.Obviously, RNN performed very well when compared to classical machine learning algorithms.
This is mainly possible because RNNs implicitly extracts the underlying features and also identifies the characteristics of the data.
This lead to better accuracy.

assistant: text: Recurrent neural networks for Cybersecurity use-cases
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep, Skinny Neural Networks are not Universal Approximators

Paper content:
In order to choose a neural network architecture that will be effective for a particular modeling problem, one must understand the limitations imposed by each of the potential options.
These limitations are typically described in terms of information theoretic bounds, or by comparing the relative complexity needed to approximate example functions between different architectures.
In this paper, we examine the topological constraints that the architecture of a neural network imposes on the level sets of all the functions that it is able to approximate.
This approach is novel for both the nature of the limitations and the fact that they are independent of network depth for a broad family of activation functions.

assistant: text: This paper proves that skinny neural networks cannot approximate certain functions, no matter how deep they are.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Activation Maximization Generative Adversarial Nets

Paper content:
Class labels have been empirically shown useful in improving the sample quality of generative adversarial nets (GANs).
In this paper, we mathematically study the properties of the current variants of GANs that make use of class label information.
With class aware gradient and cross-entropy decomposition, we reveal how class labels and associated losses influence GAN's training.
Based on that, we propose Activation Maximization Generative Adversarial Networks (AM-GAN) as an advanced solution.
Comprehensive experiments have been conducted to validate our analysis and evaluate the effectiveness of our solution, where AM-GAN outperforms other strong baselines and achieves state-of-the-art Inception Score (8.91) on CIFAR-10.
In addition, we demonstrate that, with the Inception ImageNet classifier, Inception Score mainly tracks the diversity of the generator, and there is, however, no reliable evidence that it can reflect the true sample quality.
We thus propose a new metric, called AM Score, to provide more accurate estimation on the sample quality.
Our proposed model also outperforms the baseline methods in the new metric.

assistant: text: Understand how class labels help GAN training. Propose a new evaluation metric for generative models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On the Decision Boundaries of Deep Neural Networks: A Tropical Geometry Perspective

Paper content:
This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piece-wise linear non-linearity activations.
We use tropical geometry, a new development in the area of algebraic geometry, to provide a characterization of the decision boundaries of a simple neural network of the form (Affine, ReLU, Affine).
Specifically, we show that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes.
The generators of the zonotopes are precise functions of the neural network parameters.
We utilize this geometric characterization to shed light and new perspective on three tasks.
In doing so, we propose a new tropical perspective for the lottery ticket hypothesis, where we see the effect of different initializations on the tropical geometric representation of the decision boundaries.
Also, we leverage this characterization as a new set of tropical regularizers, which  deal directly  with the decision boundaries of a network.
We investigate the use of these regularizers  in neural network pruning (removing network parameters that do not contribute to the tropical geometric representation of the decision boundaries) and in generating adversarial input attacks (with input perturbations explicitly perturbing the decision boundaries geometry to change the network prediction of the input).
In this paper, we leverage tropical geometry to characterize the decision boundaries of neural networks in the form (Affine, ReLU, Affine) and relate it to well-studied geometric objects such as zonotopes and polytopes.
We leaverage this representation in providing a tropical perspective to support the lottery ticket hypothesis, network pruning and designing adversarial attacks.
One natural extension for this work is a compact derivation for the characterization of the decision boundaries of convolutional neural networks (CNNs) and graphical convolutional networks (GCNs).
Diego Ardila, Atilla P. Kiraly, Sujeeth Bharadwaj, Bokyung Choi, Joshua J. Reicher, Lily Peng, Daniel Tse, Mozziyar Etemadi, Wenxing Ye, Greg Corrado, David P. Naidich, and Shravya Shetty.
End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography.
Nature Medicine, 2019.
A PRELIMINARIES AND DEFINITIONS.
Fact
1. P+Q = {p + q, ∀p ∈ P and q ∈ Q} is the Minkowski sum between two sets P and Q. Fact
2. Let f be a tropical polynomial and let a ∈ N. Then
Let both f and g be tropical polynomials, Then
Note that V(P(f )) is the set of vertices of the polytope P(f ).

assistant: text: Tropical geometry can be leveraged to represent the decision boundaries of neural networks and bring to light interesting insights.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Non-linear System Identification from Partial Observations via Iterative Smoothing and Learning

Paper content:
System identification is the process of building a mathematical model of an unknown system from measurements of its inputs and outputs.
It is a key step for model-based control, estimator design, and output prediction.
This work presents an algorithm for non-linear offline system identification from partial observations, i.e. situations in which the system's full-state is not directly observable.
The algorithm presented, called SISL, iteratively infers the system's full state through non-linear optimization and then updates the model parameters.
We test our algorithm on a simulated system of coupled Lorenz attractors, showing our algorithm's ability to identify high-dimensional systems that prove intractable for particle-based approaches.
We also use SISL to identify the dynamics of an aerobatic helicopter.
By augmenting the state with unobserved fluid states, we learn a model that predicts the acceleration of the helicopter better than state-of-the-art approaches.

assistant: text: This work presents a scalable algorithm for non-linear offline system identification from partial observations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generalized Clustering by Learning to Optimize Expected Normalized Cuts

Paper content:
We introduce a novel end-to-end approach for learning to cluster in the absence of labeled examples.
Our clustering objective is based on optimizing normalized cuts, a criterion which measures both intra-cluster similarity as well as inter-cluster dissimilarity.
We define a differentiable loss function equivalent to the expected normalized cuts.
Unlike much of the work in unsupervised deep learning, our trained model directly outputs final cluster assignments, rather than embeddings that need further processing to be usable.
Our approach generalizes to unseen datasets across a wide variety of domains, including text, and image.
Specifically, we achieve state-of-the-art results on popular unsupervised clustering benchmarks (e.g., MNIST, Reuters, CIFAR-10, and CIFAR-100), outperforming the strongest baselines by up to 10.9%.
Our generalization results are superior (by up to 21.9%) to the recent top-performing clustering approach with the ability to generalize.

assistant: text: We introduce a novel end-to-end approach for learning to cluster in the absence of labeled examples. We define a differentiable loss function equivalent to the expected normalized cuts.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Semi-supervised classification by reaching consensus among modalities

Paper content:
Deep learning has demonstrated abilities to learn complex structures, but they can be restricted by available data.
Recently, Consensus Networks (CNs) were proposed to alleviate data sparsity by utilizing features from multiple modalities, but they too have been limited by the size of labeled data.
In this paper, we extend CN to Transductive Consensus Networks (TCNs), suitable for semi-supervised learning.
In TCNs, different modalities of input are compressed into latent representations, which we encourage to become indistinguishable during iterative adversarial training.
To understand TCNs two mechanisms, consensus and classification, we put forward its three variants in ablation studies on these mechanisms.
To further investigate TCN models, we treat the latent representations as probability distributions and measure their similarities as the negative relative Jensen-Shannon divergences.
We show that a consensus state beneficial for classification desires a stable but imperfect similarity between the representations.
Overall, TCNs outperform or align with the best benchmark algorithms given 20 to 200 labeled samples on the Bank Marketing and the DementiaBank datasets.

assistant: text: TCN for multimodal semi-supervised learning + ablation study of its mechanisms + interpretations of latent representations
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Auto-Encoding Explanatory Examples

Paper content:
In this paper, we ask for the main factors that determine a classifier's decision making and uncover such factors by studying latent codes produced by auto-encoding frameworks.
To deliver an explanation of a classifier's behaviour, we propose a method that provides series of examples highlighting semantic differences between the classifier's decisions.
We generate these examples through interpolations in latent space.
We introduce and formalize the notion of a semantic stochastic path, as a suitable stochastic process defined in feature space via latent code interpolations.
We then introduce the concept of semantic Lagrangians as a way to incorporate the desired classifier's behaviour and find that the solution of the associated variational problem allows for highlighting differences in the classifier decision.
Very importantly, within our framework the classifier is used as a black-box, and only its evaluation is required.

assistant: text: We generate examples to explain a classifier desicion via interpolations in latent space. The variational auto encoder cost is extended with a functional of the classifier over the generated example path in data space.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Graph Topological Features via GAN

Paper content:
Inspired by the success of generative adversarial networks (GANs) in image domains, we introduce a novel hierarchical architecture for learning characteristic topological features from a single arbitrary input graph via GANs.
The hierarchical architecture consisting of multiple GANs preserves both local and global topological features, and automatically partitions the input graph into representative stages for feature learning.
The stages facilitate reconstruction and can be used as indicators of the importance of the associated topological structures.
Experiments show that our method produces subgraphs retaining a wide range of topological features, even in early reconstruction stages.
This paper contains original research on combining the use of GANs and graph topological analysis.

assistant: text: A GAN based method to learn important topological features of an arbitrary input graph.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Understanding Local Minima in Neural Networks by Loss Surface Decomposition

Paper content:
To provide principled ways of designing proper Deep Neural Network (DNN) models, it is essential to understand the loss surface of DNNs under realistic assumptions.
We introduce interesting aspects for understanding the local minima and overall structure of the loss surface.
The parameter domain of the loss surface can be decomposed into regions in which activation values (zero or one for rectified linear units) are consistent.
We found that, in each region, the loss surface have properties similar to that of linear neural networks where every local minimum is a global minimum.
This means that every differentiable local minimum is the global minimum of the corresponding region.
We prove that for a neural network with one hidden layer using rectified linear units under realistic assumptions.
There are poor regions that lead to poor local minima, and we explain why such regions exist even in the overparameterized DNNs.

assistant: text: The loss surface of neural networks is a disjoint union of regions where every local minimum is a global minimum of the corresponding region.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: iSOM-GSN: An Integrative Approach for Transforming Multi-omic Data into Gene Similarity Networks via Self-organizing Maps

Paper content:
One of the main challenges in applying graph convolutional neural networks on gene-interaction data is the lack of understanding of the vector space  to which they belong and also the inherent difficulties involved in representing those interactions on a significantly lower dimension, viz Euclidean spaces.
The challenge becomes more prevalent when dealing with various types of heterogeneous data.
We introduce a systematic, generalized method, called iSOM-GSN, used to transform ``multi-omic'' data with higher dimensions onto a two-dimensional grid.
Afterwards, we apply a convolutional neural network to predict disease states of various types.
Based on the idea of Kohonen's self-organizing map, we generate a two-dimensional grid for each sample for a given set of genes that represent a gene similarity network.
We have tested the model to predict breast and prostate cancer using gene expression, DNA methylation and copy number alteration, yielding prediction accuracies in the 94-98% range for tumor stages of breast cancer and calculated Gleason scores of prostate cancer with just 11 input genes for both cases.
The scheme not only outputs nearly perfect classification accuracy, but also provides an enhanced scheme for representation learning, visualization, dimensionality reduction, and interpretation of the results.

assistant: text: This paper presents a deep learning model that combines self-organizing maps and convolutional neural networks for representation learning of multi-omics data
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On Self Modulation for Generative Adversarial Networks

Paper content:
Training Generative Adversarial Networks (GANs) is notoriously challenging.
We propose and study an architectural modification, self-modulation, which improves GAN performance across different data sets, architectures, losses, regularizers, and hyperparameter settings.
Intuitively, self-modulation allows the intermediate feature maps of a generator to change as a function of the input noise vector.
While reminiscent of other conditioning techniques, it requires no labeled data.
In a large-scale empirical study we observe a relative decrease of 5%-35% in FID.
Furthermore, all else being equal, adding this modification to the generator leads to improved performance in 124/144 (86%) of the studied settings.
Self-modulation is a simple architectural change that requires no additional parameter tuning, which suggests that it can be applied readily to any GAN.

assistant: text: A simple GAN modification that improves performance across many losses, architectures, regularization schemes, and datasets.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning

Paper content:
Combining deep model-free reinforcement learning with on-line planning is a promising approach to building on the successes of deep RL.
On-line planning with look-ahead trees has proven successful in environments where transition models are known a priori.
However, in complex environments where transition models need to be learned from data, the deficiencies of learned models have limited their utility for planning.
To address these challenges, we propose TreeQN, a differentiable, recursive, tree-structured model that serves as a drop-in replacement for any value function network in deep RL with discrete actions.
TreeQN dynamically constructs a tree by recursively applying a transition model in a learned abstract state space and then aggregating predicted rewards and state-values using a tree backup to estimate Q-values.
We also propose ATreeC, an actor-critic variant that augments TreeQN with a softmax layer to form a stochastic policy network.
Both approaches are trained end-to-end, such that the learned model is optimised for its actual use in the tree.
We show that TreeQN and ATreeC outperform n-step DQN and A2C on a box-pushing task, as well as n-step DQN and value prediction networks (Oh et al., 2017) on multiple Atari games.
Furthermore, we present ablation studies that demonstrate the effect of different auxiliary losses on learning transition models.

assistant: text: We present TreeQN and ATreeC, new architectures for deep reinforcement learning in discrete-action domains that integrate differentiable on-line tree planning into the action-value function or policy.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: SAFE-DNN: A Deep Neural Network with Spike Assisted Feature Extraction for Noise Robust Inference

Paper content:
We present a Deep Neural Network with Spike Assisted Feature Extraction (SAFE-DNN) to improve robustness of classification under stochastic perturbation of inputs.
The proposed network augments a DNN with unsupervised learning of low-level features using spiking neuron network (SNN) with Spike-Time-Dependent-Plasticity (STDP).
The complete network learns to ignore local perturbation while performing global feature detection and classification.
The experimental results on CIFAR-10 and ImageNet subset demonstrate improved noise robustness for multiple DNN architectures without sacrificing accuracy on clean images.

assistant: text: A noise robust deep learning architecture.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Pragmatic Evaluation of Adversarial Examples in Natural Language

Paper content:
Attacks on natural language models are difficult to compare due to their different definitions of what constitutes a successful attack.
We present a taxonomy of constraints to categorize these attacks.
For each constraint, we present a real-world use case and a way to measure how well generated samples enforce the constraint.
We then employ our framework to evaluate two state-of-the art attacks which fool models with synonym substitution.
These attacks claim their adversarial perturbations preserve the semantics and syntactical correctness of the inputs, but our analysis shows these constraints are not strongly enforced.
For a significant portion of these adversarial examples, a grammar checker detects an increase in errors.
Additionally, human studies indicate that many of these adversarial examples diverge in semantic meaning from the input or do not appear to be human-written.
Finally, we highlight the need for standardized evaluation of attacks that share constraints.
Without shared evaluation metrics, it is up to researchers to set thresholds that determine the trade-off between attack quality and attack success.
We recommend well-designed human studies to determine the best threshold to approximate human judgement.

assistant: text: We present a framework for evaluating adversarial examples in natural language processing and demonstrate that generated adversarial examples are often not semantics-preserving, syntactically correct, or non-suspicious.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Seeing the whole picture instead of a single point: Self-supervised likelihood learning for deep generative models

Paper content:
Recent findings show that deep generative models can judge out-of-distribution samples as more likely than those drawn from the same distribution as the training data.
In this work, we focus on variational autoencoders (VAEs) and address the problem of misaligned likelihood estimates on image data.
We develop a novel likelihood function that is based not only on the parameters returned by the VAE but also on the features of the data learned in a self-supervised fashion.
In this way, the model additionally captures the semantic information that is disregarded by the usual VAE likelihood function.
We demonstrate the improvements in reliability of the estimates with experiments on the FashionMNIST and MNIST datasets.

assistant: text: Improved likelihood estimates in variational autoencoders using self-supervised feature learning
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Latent forward model for Real-time Strategy game planning with incomplete information

Paper content:
Model-free deep reinforcement learning approaches have shown superhuman performance in simulated environments (e.g., Atari games, Go, etc).
During training, these approaches often implicitly construct a latent space that contains key information for decision making.
In this paper, we learn a forward model on this latent space and apply it to model-based planning in miniature Real-time Strategy game with incomplete information (MiniRTS).
We first show that the latent space constructed from existing actor-critic models contains relevant information of the game, and design training procedure to learn forward models.
We also show that our learned forward model can predict meaningful future state and is usable for latent space Monte-Carlo Tree Search (MCTS), in terms of win rates against rule-based agents.

assistant: text: The paper analyzes the latent space learned by model-free approaches in a miniature incomplete information game, trains a forward model in the latent space and apply it to Monte-Carlo Tree Search, yielding positive performance.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Modeling question asking using neural program generation

Paper content:
People ask questions that are far richer, more informative, and more creative than current AI systems.
We propose a neural program generation framework for modeling human question asking, which represents questions as formal programs and generates programs with an encoder-decoder based deep neural network.
From extensive experiments using an information-search game, we show that our method can ask optimal questions in synthetic settings, and predict which questions humans are likely to ask in unconstrained settings.
We also propose a novel grammar-based question generation framework trained with reinforcement learning, which is able to generate creative questions without supervised data.

assistant: text: We introduce a model of human question asking that combines neural networks and symbolic programs, which can learn to generate good questions with or without supervised examples.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Sentence embedding with contrastive multi-views learning

Paper content:
In this work, we propose a self-supervised method to learn sentence representations with an injection of linguistic knowledge.
Multiple linguistic frameworks propose diverse sentence structures from which semantic meaning might be expressed out of compositional words operations.
We aim to take advantage of this linguist diversity and learn to represent sentences by contrasting these diverse views.
Formally, multiple views of the same sentence are mapped to close representations.
On the contrary, views from other sentences are mapped further.
By contrasting different linguistic views, we aim at building embeddings which better capture semantic and which are less sensitive to the sentence outward form.

assistant: text: We aim to exploit the diversity of linguistic structures to build sentence representations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
Emotion is playing a great role in our daily lives.
The necessity and importance of an automatic Emotion recognition system is getting increased.
Traditional approaches of emotion recognition are based on facial images, measurements of heart rates, blood pressure, temperatures, tones of voice/speech, etc.
However, these features can potentially be changed to fake features.
So to detect hidden and real features that is not controlled by the person are data measured from brain signals.
There are various ways of measuring brain waves: EEG, MEG, FMRI, etc.
On the bases of cost effectiveness and performance trade-offs, EEG is chosen for emotion recognition in this work.
The main aim of this study is to detect emotion based on EEG signal analysis recorded from brain in response to visual stimuli.
The approaches used were the selected visual stimuli were presented to 11 healthy target subjects and EEG signal were recorded in controlled situation to minimize artefacts (muscle or/and eye movements).
The signals were filtered and type of frequency band was computed and detected.
The proposed method predicts an emotion type (positive/negative) in response to the presented stimuli.
Finally, the performance of the proposed approach was tested.
The average accuracy of machine learning algorithms (i.e. J48, Bayes Net, Adaboost and Random Forest) are 78.86, 74.76, 77.82 and 82.46 respectively.
In this study, we also applied EEG applications in the context of neuro-marketing.
The results empirically demonstrated detection of the favourite colour preference of customers in response to the logo colour of an organization or Service.
Emotion is playing a great role in our daily lives.
The necessity and importance of an automatic 25 Emotion recognition system is getting increased.
Traditional approaches of emotion recognition 26 are based on facial images, measurements of heart rates, blood pressure, temperatures, tones of 27 voice/speech, etc.
However, these features can potentially be changed to fake features.
Thus, to simple and portable device [2] .
The brainwave activity is broadly divided into five frequency bands.
The boundary between the frequency bands is not strict but not varying much.
The frequency bands 63 include delta(0.5-4Hz), theta(5-8Hz), alpha(9-12Hz), beta(13-30Hz) and gamma(above 30Hz) [3] .
For this study, EEG data is collected using Emotiv EPOC device with 14 electrodes located at AF3, gamma band is also responsible for arousal [6] .
In other words, the EEG brain activity from parietal 90 and frontal lobe of the brain is more emotionally informative where as gamma, alpha and beta waves
We collected and prepared three image data sets for stimuli presentation and classification.
These
To answer research question 1 and 2, the top ranked features for each subject are extracted using

assistant: text: This paper presents EEG based emotion detection of a person towards an image stimuli and its applicability on neuromarketing.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Why do deep convolutional networks generalize so poorly to small image transformations?

Paper content:
Deep convolutional network architectures are often assumed to guarantee generalization for small image translations and deformations.
In this paper we show that modern CNNs (VGG16, ResNet50, and InceptionResNetV2) can drastically change their output when an image is translated in the image plane by a few pixels, and that this failure of generalization also happens with other realistic small image transformations.
Furthermore,  we see these failures to generalize more frequently in more modern networks.
We show that these failures are related to the fact that the architecture of modern CNNs ignores the classical sampling theorem so that generalization is not guaranteed.
We also show that biases in the statistics of commonly used image datasets makes it unlikely that CNNs will learn to be invariant to these transformations.
Taken together our results suggest that the performance of CNNs in object recognition falls far short of the generalization capabilities of humans.

assistant: text: Modern deep CNNs are not invariant to translations, scalings and other realistic image transformations, and this lack of invariance is related to the subsampling operation and the biases contained in image datasets.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering

Paper content:
Conversational question answering (CQA) is a novel QA task that requires the understanding of dialogue context.
Different from traditional single-turn machine reading comprehension (MRC), CQA is a comprehensive task comprised of passage reading, coreference resolution, and contextual understanding.
In this paper, we propose an innovative contextualized attention-based deep neural network, SDNet, to fuse context into traditional MRC models.
Our model leverages both inter-attention and self-attention to comprehend the conversation and passage.
Furthermore, we demonstrate a novel method to integrate the BERT contextual model as a sub-module in our network.
Empirical results show the effectiveness of SDNet.
On the CoQA leaderboard, it outperforms the previous best model's F1 score by 1.6%.
Our ensemble model further improves the F1 score by 2.7%.

assistant: text: A neural method for conversational question answering with attention mechanism and a novel usage of BERT as contextual embedder
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: How well do deep neural networks trained on object recognition characterize the mouse visual system?

Paper content:
Recent work on modeling neural responses in the primate visual system has benefited from deep neural networks trained on large-scale object recognition, and found a hierarchical correspondence between layers of the artificial neural network and brain areas along the ventral visual stream.
However, we neither know whether such task-optimized networks enable equally good models of the rodent visual system, nor if a similar hierarchical correspondence exists.
Here, we address these questions in the mouse visual system by extracting features at several layers of a convolutional neural network (CNN) trained on ImageNet to predict the responses of thousands of neurons in four visual areas (V1, LM, AL, RL) to natural images.
We found that the CNN features outperform classical subunit energy models, but found no evidence for an order of the areas we recorded via a correspondence to the hierarchy of CNN layers.
Moreover, the same CNN but with random weights provided an equivalently useful feature space for predicting neural responses.
Our results suggest that object recognition as a high-level task does not provide more discriminative features to characterize the mouse visual system than a random network.
Unlike in the primate, training on ethologically relevant visually guided behaviors -- beyond static object recognition -- may be needed to unveil the functional organization of the mouse visual cortex.

assistant: text: A goal-driven approach to model four mouse visual areas (V1, LM, AL, RL) based on deep neural networks trained on static object recognition does not unveil a functional organization of visual cortex unlike in primates
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Expensive Coordination: An Event-Based Deep RL Approach

Paper content:
Existing works in deep Multi-Agent Reinforcement Learning (MARL) mainly focus on coordinating cooperative agents to complete certain tasks jointly.
However, in many cases of the real world, agents are self-interested such as employees in a company and clubs in a league.
Therefore, the leader, i.e., the manager of the company or the league, needs to provide bonuses to followers for efficient coordination, which we call expensive coordination.
The main difficulties of expensive coordination are that
i) the leader has to consider the long-term effect and predict the followers' behaviors when assigning bonuses and
ii) the complex interactions between followers make the training process hard to converge, especially when the leader's policy changes with time.
In this work, we address this problem through an event-based deep RL approach.
Our main contributions are threefold.
(1) We model the leader's decision-making process as a semi-Markov Decision Process and propose a novel multi-agent event-based policy gradient to learn the leader's long-term policy.
(2) We exploit the leader-follower consistency scheme to design a follower-aware module and a follower-specific attention module to predict the followers' behaviors and make accurate response to their behaviors.
(3) We propose an action abstraction-based policy gradient algorithm to reduce the followers' decision space and thus accelerate the training process of followers.
Experiments in resource collections, navigation, and the predator-prey game reveal that our approach outperforms the state-of-the-art methods dramatically.

assistant: text: We propose an event-based policy gradient  to train the leader and an action abstraction policy gradient to train the followers in leader-follower Markov game.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Co-Generation with GANs using AIS based HMC

Paper content:
Inferring the most likely configuration for a subset of variables of a joint distribution given the remaining ones – which we refer to as co-generation – is an important challenge that is computationally demanding for all but the simplest settings.
This task has received a considerable amount of attention, particularly for classical ways of modeling distributions like structured prediction.
In contrast, almost nothing is known about this task when considering recently proposed techniques for modeling high-dimensional distributions, particularly generative adversarial nets (GANs).
Therefore, in this paper, we study the occurring challenges for co-generation with GANs.
To address those challenges we develop an annealed importance sampling (AIS) based Hamiltonian Monte Carlo (HMC) co-generation algorithm.
The presented approach significantly outperforms classical gradient-based methods on synthetic data and on CelebA.

assistant: text: Using annealed importance sampling on the co-generation problem.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Binarized Back-Propagation: Training Binarized Neural Networks with Binarized Gradients

Paper content:
Binarized Neural networks (BNNs) have been shown to be effective in improving network efficiency during the inference phase, after the network has been trained.
However, BNNs only binarize the model parameters and activations during propagations.
Therefore, BNNs do not offer significant efficiency improvements during training, since the gradients are still propagated and used with high precision.
We show there is no inherent difficulty in training BNNs using "Binarized BackPropagation" (BBP), in which we also binarize the gradients.
To avoid significant degradation in test accuracy, we simply increase the number of filter maps in a each convolution layer.
Using BBP on dedicated hardware can potentially significantly improve the execution efficiency (\emph{e.g.}, reduce dynamic memory footprint, memory bandwidth and computational energy) and speed up the training process with an appropriate hardware support, even after such an increase in network size.
Moreover, our method is ideal for distributed learning as it reduces the communication costs significantly (e.g., by ~32).
Using this method, we demonstrate a minimal loss in classification accuracy on several datasets and topologies.

assistant: text: Binarized Back-Propagation all you need for completely binarized training is to is to inflate the size of the network
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learnability for the Information Bottleneck

Paper content:
Compressed representations generalize better (Shamir et al., 2010), which may be crucial when learning from limited or noisy labeled data.
The Information Bottleneck (IB) method (Tishby et al. (2000)) provides an insightful and principled approach for balancing compression and prediction in representation learning.
The IB objective I(X; Z) − βI(Y ; Z) employs a Lagrange multiplier β to tune this trade-off.
However, there is little theoretical guidance for how to select β.
There is also a lack of theoretical understanding about the relationship between β, the dataset, model capacity, and learnability.
In this work, we show that if β is improperly chosen, learning cannot happen: the trivial representation P(Z|X) = P(Z) becomes the global minimum of the IB objective.
We show how this can be avoided, by identifying a sharp phase transition between the unlearnable and the learnable which arises as β varies.
This phase transition defines the concept of IB-Learnability.
We prove several sufficient conditions for IB-Learnability, providing theoretical guidance for selecting β.
We further show that IB-learnability is determined by the largest confident, typical, and imbalanced subset of the training examples.
We give a practical algorithm to estimate the minimum β for a given dataset.
We test our theoretical results on synthetic datasets, MNIST, and CIFAR10 with noisy labels, and make the surprising observation that accuracy may be non-monotonic in β.

assistant: text: Theory predicts the phase transition between unlearnable and learnable values of beta for the Information Bottleneck objective
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Online Hyperparameter Adaptation via Amortized Proximal Optimization

Paper content:
Effective performance of neural networks depends critically on effective tuning of optimization hyperparameters, especially learning rates (and schedules thereof).
We present Amortized Proximal Optimization (APO), which takes the perspective that each optimization step should approximately minimize a proximal objective (similar to the ones used to motivate natural gradient and trust region policy optimization).
Optimization hyperparameters are adapted to best minimize the proximal objective after one weight update.
We show that an idealized version of APO (where an oracle minimizes the proximal objective exactly) achieves global convergence to stationary point and locally second-order convergence to global optimum for neural networks.
APO incurs minimal computational overhead.
We experiment with using APO to adapt a variety of optimization hyperparameters online during training, including (possibly layer-specific) learning rates, damping coefficients, and gradient variance exponents.
For a variety of network architectures and optimization algorithms (including SGD, RMSprop, and K-FAC), we show that with minimal tuning, APO performs competitively with carefully tuned optimizers.

assistant: text: We introduce amortized proximal optimization (APO), a method to adapt a variety of optimization hyperparameters online during training, including learning rates, damping coefficients, and gradient variance exponents.