ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Incremental training of multi-generative adversarial networks

Paper content:
Generative neural networks map a standard, possibly distribution to a complex high-dimensional distribution, which represents the real world data set.
However, a determinate input distribution as well as a specific architecture of neural networks may impose limitations on capturing the diversity in the high dimensional target space.
To resolve this difficulty, we propose a training framework that greedily produce a series of generative adversarial networks that incrementally capture the diversity of the target space.
We show theoretically and empirically that our training algorithm converges to the theoretically optimal distribution, the projection of the real distribution onto the convex hull of the network's distribution space.

assistant: text: We propose a new method to incrementally train a mixture generative model to approximate the information projection of the real data distribution.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards Holistic and Automatic Evaluation of Open-Domain Dialogue Generation

Paper content:
Open-domain dialogue generation has gained increasing attention in Natural Language Processing.
Comparing these methods requires a holistic means of dialogue evaluation.
Human ratings are deemed as the gold standard.
As human evaluation is inefficient and costly, an automated substitute is desirable.
In this paper, we propose holistic evaluation metrics which capture both the quality and diversity of dialogues.
Our metrics consists of (1) GPT-2 based context coherence between sentences in a dialogue, (2) GPT-2 based fluency in phrasing, and, (3) $n$-gram based diversity in responses to augmented queries.
The empirical validity of our metrics is demonstrated by strong correlation with human judgments.
We provide the associated code, datasets and human ratings.
This paper provides a holistic and automatic evaluation method of open-domain dialogue models.
In contrast to prior art, our means of evaluation captures not only the quality of generation, but also the diversity of responses.
We recruit GPT-2 as a strong language model to evaluate the fluency and context-coherency of a dialogue.
For diversity evaluation, the diversity of queries is controlled while the diversity of responses is evaluated by n-gram entropy.
Two methods for controlled diversity are proposed, WordNet Substitution and Conditional Text Generator.
The proposed metrics show strong correlation with human judgments.
We are providing the implementations of our proposed metrics, associated fine-tuned models and datasets to accelerate the research on open-domain dialogue systems.
It is our hope the proposed holistic metrics may pave the way towards comparability of open-domain dialogue methods.

assistant: text: We propose automatic metrics to holistically evaluate open-dialogue generation and they strongly correlate with human evaluation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: From Amortised to Memoised Inference: Combining Wake-Sleep and Variational-Bayes for Unsupervised Few-Shot Program Learning

Paper content:
Given a large database of concepts but only one or a few examples of each, can we learn models for each concept that are not only generalisable, but interpretable?
In this work, we aim to tackle this problem through hierarchical Bayesian program induction.
We present a novel learning algorithm which can infer concepts as short, generative, stochastic programs, while learning a global prior over programs to improve generalisation and a recognition network for efficient inference.
Our algorithm, Wake-Sleep-Remember (WSR), combines gradient learning for continuous parameters with neurally-guided search over programs.
We show that WSR learns compelling latent programs in two tough symbolic domains: cellular automata and Gaussian process kernels.
We also collect and evaluate on a new dataset, Text-Concepts, for discovering structured patterns in natural text data.

assistant: text: We extend the wake-sleep algorithm and use it to learn to learn structured models from few examples,
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning Covariate-Specific Embeddings with Tensor Decompositions

Paper content:
Word embedding is a useful approach to capture co-occurrence structures in a large corpus of text.
In addition to the text data itself, we often have additional covariates associated with individual documents in the corpus---e.g. the demographic of the author, time and venue of publication, etc.---and we would like the embedding to naturally capture the information of the covariates.
In this paper, we propose a new tensor decomposition model for word embeddings with covariates.
Our model jointly learns a \emph{base} embedding for all the words as well as a weighted diagonal transformation to model how each covariate modifies the base embedding.
To obtain the specific embedding for a particular author or venue, for example, we can then simply multiply the base embedding by the transformation matrix associated with that time or venue.
The main advantages of our approach is data efficiency and interpretability of the covariate transformation matrix.
Our experiments demonstrate that our joint model learns substantially better embeddings conditioned on each covariate compared to the standard approach of learning a separate embedding for each covariate using only the relevant subset of data.
Furthermore, our model encourages the embeddings to be ``topic-aligned'' in the sense that the dimensions have specific independent meanings.
This allows our covariate-specific embeddings to be compared by topic, enabling downstream differential analysis.
We empirically evaluate the benefits of our algorithm on several datasets, and demonstrate how it can be used to address many natural questions about the effects of covariates.

assistant: text: Using the same embedding across covariates doesn't make sense, we show that a tensor decomposition algorithm learns sparse covariate-specific embeddings and naturally separable topics jointly and data-efficiently.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Batch Normalization is a Cause of Adversarial Vulnerability

Paper content:
Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks.
In many cases it indeed decreases the number of parameter updates required to achieve low training error.
However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard datasets.
Furthermore, substituting weight decay for batch norm is sufficient to nullify the relationship between adversarial vulnerability and the input dimension.
Our work is consistent with a mean-field analysis that found that batch norm causes exploding gradients.
Batch norm is a standard component of modern deep neural networks, and tends to make the training process less sensitive to the choice of hyperparameters in many cases (Ioffe & Szegedy, 2015) .
While ease of training is desirable for model developers, an important concern among stakeholders is that of model robustness to plausible, previously unseen inputs during deployment.The adversarial examples phenomenon has exposed unstable predictions across state-of-the-art models (Szegedy et al., 2014) .
This has led to a variety of methods that aim to improve robustness, but doing so effectively remains a challenge BID0 Schott et al., 2019; Hendrycks & Dietterich, 2019; Jacobsen et al., 2019) .
We believe that a prerequisite to developing methods that increase robustness is an understanding of factors that reduce it.Presented at the ICML 2019 Workshop on Identifying and Understanding Deep Learning Phenomena.
Copyright 2019 by the author(s).
We found that there is no free lunch with batch norm: the accelerated training properties and occasionally higher clean test accuracy come at the cost of robustness, both to additive noise and for adversarial perturbations.
We have shown that there is no inherent relationship between the input dimension and vulnerability.
Our results highlight the importance of identifying the disparate mechanisms of regularization techniques, especially when concerned about robustness.Bjorck, N., Gomes, C. P., Selman, B., and Weinberger, K. Q.Understanding

assistant: text: Batch normalization reduces adversarial robustness, as well as general robustness in many cases, particularly to noise corruptions.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Evaluating The Search Phase of Neural Architecture Search

Paper content:
Neural Architecture Search (NAS) aims to facilitate the design of deep networks for new tasks.
Existing techniques rely on two stages: searching over the architecture space and validating the best architecture.
NAS algorithms are currently compared solely based on their results on the downstream task.
While intuitive, this fails to explicitly evaluate the effectiveness of their search strategies.
In this paper, we propose to evaluate the NAS search phase.
To this end, we compare the quality of the solutions obtained by NAS search policies with that of random architecture selection.
We find that:
(i) On average, the state-of-the-art NAS algorithms perform similarly to the random policy;
(ii) the widely-used weight sharing strategy degrades the ranking of the NAS candidates to the point of not reflecting their true performance, thus reducing the effectiveness of the search process.
We believe that our evaluation framework will be key to designing NAS strategies that consistently discover architectures superior to random ones.

assistant: text: We empirically disprove a fundamental hypothesis of the widely-adopted weight sharing strategy in neural architecture search and explain why the state-of-the-arts NAS algorithms performs similarly to random search.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Verification of Generative-Model-Based Visual Transformations

Paper content:
Generative networks are promising models for specifying visual transformations.
Unfortunately, certification of generative models is challenging as one needs to capture sufficient non-convexity so to produce precise bounds on the output.
Existing verification methods either fail to scale to generative networks or do not capture enough non-convexity.
In this work, we present a new verifier, called ApproxLine, that can certify non-trivial properties of generative networks.
ApproxLine performs both deterministic and probabilistic abstract interpretation and captures infinite sets of outputs of generative networks.
We show that ApproxLine can verify interesting interpolations in the network's latent space.

assistant: text: We verify deterministic and probabilistic properties of neural networks using non-convex relaxations over visible transformations specified by generative models
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: PACT: Parameterized Clipping Activation for Quantized Neural Networks

Paper content:
Deep learning algorithms achieve high classification accuracy at the expense of significant computation cost.
To address this cost, a number of quantization schemeshave been proposed - but most of these techniques focused on quantizing weights, which are relatively smaller in size compared to activations.
This paper proposes a novel quantization scheme for activations during training - that enables neural networks to work well with ultra low precision weights and activations without any significant accuracy degradation.
This technique, PArameterized Clipping acTi-vation (PACT), uses an activation clipping parameter α that is optimized duringtraining to find the right quantization scale.
PACT allows quantizing activations toarbitrary bit precisions, while achieving much better accuracy relative to publishedstate-of-the-art quantization schemes.
We show, for the first time, that both weights and activations can be quantized to 4-bits of precision while still achieving accuracy comparable to full precision networks across a range of popular models and datasets.
We also show that exploiting these reduced-precision computational units in hardware can enable a super-linear improvement in inferencing performance dueto a significant reduction in the area of accelerator compute engines coupled with the ability to retain the quantized model and activation data in on-chip memories.

assistant: text: A new way of quantizing activation of Deep Neural Network via parameterized clipping which optimizes the quantization scale via stochastic gradient descent.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines

Paper content:
Policy gradient methods have enjoyed great success in deep reinforcement learning but suffer from high variance of gradient estimates.
The high variance problem is particularly exasperated in problems with long horizons or high-dimensional action spaces.
To mitigate this issue, we derive a bias-free action-dependent baseline for variance reduction which fully exploits the structural form of the stochastic policy itself and does not make any additional assumptions about the MDP.
We demonstrate and quantify the benefit of the action-dependent baseline through both theoretical analysis as well as numerical results, including an analysis of the suboptimality of the optimal state-dependent baseline.
The result is a computationally efficient policy gradient algorithm, which scales to high-dimensional control problems, as demonstrated by a synthetic 2000-dimensional target matching task.
Our experimental results indicate that action-dependent baselines allow for faster learning on standard reinforcement learning benchmarks and high-dimensional hand manipulation and synthetic tasks.
Finally, we show that the general idea of including additional information in baselines for improved variance reduction can be extended to partially observed and multi-agent tasks.

assistant: text: Action-dependent baselines can be bias-free and yield greater variance reduction than state-only dependent baselines for policy gradient methods.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Backprop with Approximate Activations for Memory-efficient Network Training

Paper content:
With innovations in architecture design, deeper and wider neural network models deliver improved performance on a diverse variety of tasks.
But the increased memory footprint of these models presents a challenge during training, when all intermediate layer activations need to be stored for back-propagation.
Limited GPU memory forces practitioners to make sub-optimal choices: either train inefficiently with smaller batches of examples; or limit the architecture to have lower depth and width, and fewer layers at higher spatial resolutions.
This work introduces an approximation strategy that significantly reduces a network's memory footprint during training, but has negligible effect on training performance and computational expense.
During the forward pass, we replace activations with lower-precision approximations immediately after they have been used by subsequent layers, thus freeing up memory.
The approximate activations are then used during the backward pass.
This approach limits the accumulation of errors across the forward and backward pass---because the forward computation across the network still happens at full precision, and the approximation has a limited effect when computing gradients to a layer's input.
Experiments, on CIFAR and ImageNet, show that using our approach with 8- and even 4-bit fixed-point approximations of 32-bit floating-point activations has only a minor effect on training and validation performance, while affording significant savings in memory usage.

assistant: text: An algorithm to reduce the amount of memory required for training deep networks, based on an approximation strategy.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Hierarchical Adversarially Learned Inference

Paper content:
We propose a novel hierarchical generative model with a simple Markovian structure and a corresponding inference model.
Both the generative and inference model are trained using the adversarial learning paradigm.
We demonstrate that the hierarchical structure supports the learning of progressively more abstract representations as well as providing semantically meaningful reconstructions with different levels of fidelity.
Furthermore, we show that minimizing the Jensen-Shanon divergence between the generative and inference network is enough to minimize the reconstruction error.
The resulting semantically meaningful hierarchical latent structure discovery is exemplified on the CelebA dataset
.  There, we show that the features learned by our model in an unsupervised way outperform the best handcrafted features
. Furthermore, the extracted features remain competitive when compared to several recent deep supervised approaches on an attribute prediction task on CelebA.
Finally, we leverage the model's inference network to achieve state-of-the-art performance on a semi-supervised variant of the MNIST digit classification task.

assistant: text: Adversarially trained hierarchical generative model with robust and semantically learned latent representation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Data Statements for NLP: Toward Mitigating System Bias and Enabling Better Science

Paper content:
In this paper, we propose data statements as a design solution and professional practice for natural language processing technologists, in both research and development — through the adoption and widespread use of data statements, the field can begin to address critical scientific and ethical issues that result from the use of data from certain populations in the development of technology for other populations.
We present a form that data statements can take and explore the implications of adopting them as part of regular practice.
We argue that data statements will help alleviate issues related to exclusion and bias in language technology; lead to better precision in claims about how NLP research can generalize and thus better engineering results; protect companies from public embarrassment; and ultimately lead to language technology that meets its users in their own preferred linguistic style and furthermore does not mis- represent them to others.
** To appear in TACL **

assistant: text: A practical proposal for more ethical and responsive NLP technology, operationalizing transparency of test and training data
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Data Enrichment: Multi-task Learning in High Dimension with Theoretical Guarantees

Paper content:
Given samples from a group of related regression tasks, a data-enriched model describes observations by a common and per-group individual parameters.
In high-dimensional regime, each parameter has its own structure such as sparsity or group sparsity.
In this paper, we consider the general form of data enrichment where data comes in a fixed but arbitrary number of tasks $G$ and any convex function, e.g., norm, can characterize the structure of both common and individual parameters.
We propose an estimator for the high-dimensional data enriched model and investigate its statistical properties.
We delineate the sample complexity of our estimator and provide high probability non-asymptotic bound for estimation error of all parameters under a condition weaker than the state-of-the-art.
We propose an iterative estimation algorithm with a geometric convergence rate.
Overall, we present a first through statistical and computational analysis of inference in the data enriched model.

assistant: text: We provide an estimator and an estimation algorithm for a class of multi-task regression problem and provide statistical and computational analysis..
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Beyond Word Importance:  Contextual Decomposition to Extract Interactions from LSTMs

Paper content:
The driving force behind the recent success of LSTMs has been their ability to learn complex and non-linear relationships.
Consequently, our inability to describe these relationships has led to LSTMs being characterized as black boxes.
To this end, we introduce contextual decomposition (CD), an interpretation algorithm for analysing individual predictions made by standard LSTMs, without any changes to the underlying model.
By decomposing the output of a LSTM, CD captures the contributions of combinations of words or variables to the final prediction of an LSTM.
On the task of sentiment analysis with the Yelp and SST data sets, we show that CD is able to reliably identify words and phrases of contrasting sentiment, and how they are combined to yield the LSTM's final prediction.
Using the phrase-level labels in SST, we also demonstrate that CD is able to successfully extract positive and negative negations from an LSTM, something which has not previously been done.

assistant: text: We introduce contextual decompositions, an interpretation algorithm for LSTMs capable of extracting word, phrase and interaction-level importance score
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Skip-connection and batch-normalization improve data separation ability

Paper content:
The ResNet and the batch-normalization (BN) achieved high performance even when only a few labeled data are available.
However, the reasons for its high performance are unclear.
To clear the reasons, we analyzed the effect of the skip-connection in ResNet and the BN on the data separation ability, which is an important ability for the classification problem.
Our results show that, in the multilayer perceptron with randomly initialized weights, the angle between two input vectors converges to zero in an exponential order of its depth, that the skip-connection makes this exponential decrease into a sub-exponential decrease, and that the BN relaxes this sub-exponential decrease into a reciprocal decrease.
Moreover, our analysis shows that the preservation of the angle at initialization encourages trained neural networks to separate points from different classes.
These imply that the skip-connection and the BN improve the data separation ability and achieve high performance even when only a few labeled data are available.

assistant: text: The Skip-connection in ResNet and the batch-normalization improve the data separation ability and help to train a deep neural network.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: BatchEnsemble: an Alternative Approach to Efficient Ensemble and Lifelong Learning

Paper content:
Ensembles, where multiple neural networks are trained individually and their predictions are averaged, have been shown to be widely successful for improving both the accuracy and predictive uncertainty of single neural networks.
However, an ensemble's cost for both training and testing increases linearly with the number of networks.
In this paper, we propose BatchEnsemble, an ensemble method whose computational and memory costs are significantly lower than typical ensembles.
BatchEnsemble achieves this by defining each weight matrix to be the Hadamard product of a shared weight among all ensemble members and a rank-one matrix per member.
Unlike ensembles, BatchEnsemble is not only parallelizable across devices, where one device trains one member, but also parallelizable within a device, where multiple ensemble members are updated simultaneously for a given mini-batch.
Across CIFAR-10, CIFAR-100, WMT14 EN-DE/EN-FR translation, and contextual bandits tasks, BatchEnsemble yields competitive accuracy and uncertainties as typical ensembles; the speedup at test time is 3X and memory reduction is 3X at an ensemble of size 4.
We also apply BatchEnsemble to lifelong learning, where on Split-CIFAR-100, BatchEnsemble yields comparable performance to progressive neural networks while having a much lower computational and memory costs.
We further show that BatchEnsemble can easily scale up to lifelong learning on Split-ImageNet which involves 100 sequential learning tasks.

assistant: text: We introduced BatchEnsemble, an efficient method for ensembling and lifelong learning which can be used to improve the accuracy and uncertainty of any neural network like typical ensemble methods.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Baseline-corrected space-by-time non-negative matrix factorization for decoding single trial population spike trains

Paper content:
Activity of populations of sensory neurons carries stimulus information in both the temporal and the spatial dimensions.
This poses the question of how to compactly represent all the information that the population codes carry across all these dimensions.
Here, we developed an analytical method to factorize a large number of retinal ganglion cells' spike trains into a robust low-dimensional representation that captures efficiently both their spatial and temporal information.
In particular, we extended previously used single-trial space-by-time tensor decomposition based on non-negative matrix factorization to efficiently discount pre-stimulus baseline activity.
On data recorded from retinal ganglion cells with strong pre-stimulus baseline, we showed that in situations were the stimulus elicits a strong change in firing rate, our extensions yield a boost in stimulus decoding performance.
Our results thus suggest that taking into account the baseline can be important for finding a compact information-rich representation of neural activity.

assistant: text: We extended single-trial space-by-time tensor decomposition based on non-negative matrix factorization to efficiently discount pre-stimulus baseline activity that improves decoding performance on data with non-negligible baselines.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: The role of over-parametrization in generalization of neural networks

Paper content:
Despite existing work on ensuring generalization of neural networks in terms of scale sensitive complexity measures, such as norms, margin and sharpness, these complexity measures do not offer an explanation of why neural networks generalize better with over-parametrization.
In this work we suggest a novel complexity measure based on unit-wise capacities resulting in a tighter generalization bound for two layer ReLU networks.
Our capacity bound correlates with the behavior of test error with increasing network sizes (within the range reported in the experiments), and could partly explain the improvement in generalization with over-parametrization.
We further present a matching lower bound for the Rademacher complexity that improves over previous capacity lower bounds for neural networks.

assistant: text: We suggest a generalization bound that could partly explain the improvement in generalization with over-parametrization.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Stacking for Transfer Learning

Paper content:
In machine learning tasks, overtting frequently crops up when the number of samples of target domain is insufﬁcient, for the generalization ability of the classiﬁer is poor in this circumstance.
To solve this problem, transfer learning utilizes the knowledge of similar domains to improve the robustness of the learner.
The main idea of existing transfer learning algorithms is to reduce the dierence between domains by sample selection or domain adaptation.
However, no matter what transfer learning algorithm we use, the difference always exists and the hybrid training of source and target data leads to reducing ﬁtting capability of the learner on target domain.
Moreover, when the relatedness between domains is too low, negative transfer is more likely to occur.
To tackle the problem, we proposed a two-phase transfer learning architecture based on ensemble learning, which uses the existing transfer learning algorithms to train the weak learners in the ﬁrst stage, and uses the predictions of target data to train the ﬁnal learner in the second stage.
Under this architecture, the ﬁtting capability and generalization capability can be guaranteed at the same time.
We evaluated the proposed method on public datasets, which demonstrates the effectiveness and robustness of our proposed method.

assistant: text: How to use stacked generalization to improve the performance of existing transfer learning algorithms when limited labeled data is available.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: The Limitations of Adversarial Training and the Blind-Spot Attack

Paper content:
The adversarial training procedure proposed by Madry et al. (2018) is one of the most effective methods to defend against adversarial examples in deep neural net- works (DNNs).
In our paper, we shed some lights on the practicality and the hardness of adversarial training by showing that the effectiveness (robustness on test set) of adversarial training has a strong correlation with the distance between a test point and the manifold of training data embedded by the network.
Test examples that are relatively far away from this manifold are more likely to be vulnerable to adversarial attacks.
Consequentially, an adversarial training based defense is susceptible to a new class of attacks, the “blind-spot attack”, where the input images reside in “blind-spots” (low density regions) of the empirical distri- bution of training data but is still on the ground-truth data manifold.
For MNIST, we found that these blind-spots can be easily found by simply scaling and shifting image pixel values.
Most importantly, for large datasets with high dimensional and complex data manifold (CIFAR, ImageNet, etc), the existence of blind-spots in adversarial training makes defending on any valid test examples difficult due to the curse of dimensionality and the scarcity of training data.
Additionally, we find that blind-spots also exist on provable defenses including (Kolter & Wong, 2018) and (Sinha et al., 2018) because these trainable robustness certificates can only be practically optimized on a limited set of training data.

assistant: text: We show that even the strongest adversarial training methods cannot defend against adversarial examples crafted on slightly scaled and shifted test images.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Aging Memories Generate More Fluent Dialogue Responses with Memory Networks

Paper content:
The integration of a  Knowledge Base (KB) into a neural dialogue agent is one of the key challenges in Conversational AI.
Memory networks has proven to be effective to encode KB information into an external memory to thus generate more fluent and informed responses.
Unfortunately, such memory becomes full of latent representations during training, so the most common strategy is to overwrite old memory entries randomly.
In this paper, we question this approach and provide experimental evidence showing that conventional memory networks generate many redundant latent vectors resulting in overfitting and the need for larger memories.
We introduce memory dropout as an automatic technique that encourages diversity in the latent space by
1) Aging redundant memories to increase their probability of being overwritten during training
2) Sampling new memories that summarize the knowledge acquired by redundant memories.
This technique allows us to incorporate  Knowledge Bases to achieve state-of-the-art dialogue generation in the Stanford Multi-Turn Dialogue dataset.
Considering the same architecture, its use provides an improvement of +2.2 BLEU points for the automatic  generation of responses and an increase of +8.1% in the recognition of named entities.

assistant: text: Conventional memory networks generate many redundant latent vectors resulting in overfitting and the need for larger memories. We introduce memory dropout as an automatic technique that encourages diversity in the latent space.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Attacking Binarized Neural Networks

Paper content:
Neural networks with low-precision weights and activations offer compelling
efficiency advantages over their full-precision equivalents.
The two most
frequently discussed benefits of quantization are reduced memory consumption,
and a faster forward pass when implemented with efficient bitwise
operations.
We propose a third benefit of very low-precision neural networks:
improved robustness against some adversarial attacks, and in the worst case,
performance that is on par with full-precision models.
We focus on the very
low-precision case where weights and activations are both quantized to $\pm$1,
and note that stochastically quantizing weights in just one layer can sharply
reduce the impact of iterative attacks.
We observe that non-scaled binary neural
networks exhibit a similar effect to the original \emph{defensive distillation}
procedure that led to \emph{gradient masking}, and a false notion of security.
We address this by conducting both black-box and white-box experiments with
binary models that do not artificially mask gradients.

assistant: text: We conduct adversarial attacks against binarized neural networks and show that we reduce the impact of the strongest attacks, while maintaining comparable accuracy in a black-box setting
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: epsilon-Rotation Invariant Euclidean Spheres Packing in Slicer3D

Paper content:
Sometimes SRS (Stereotactic Radio Surgery) requires using sphere packing on a Region of Interest (ROI) such as cancer to determine a treatment plan.
We have developed a sphere packing algorithm which packs non-intersecting spheres inside the ROI.
The region of interest in our case are those voxels which are identified as cancer tissues.
In this paper, we analyze the rotational invariant properties of  our sphere-packing algorithm which is based on distance transformations.
Epsilon-Rotation invariant means the ability to arbitrary rotate the 3D ROI while keeping the volume properties remaining (almost) same within some limit of epsilon.
The applied rotations produce spherical packing which remains highly correlated as we analyze the geometrically properties of sphere packing before and after the rotation of the volume data for the ROI.
Our novel sphere packing algorithm has high degree of rotation invariance within the range of +/- epsilon.
Our method used a shape descriptor derived from the values of the disjoint set of spheres form the distance-based sphere packing algorithm to extract the invariant descriptor from the ROI.
We demonstrated by implementing these ideas using Slicer3D platform available for our research.
The data is based on sing MRI Stereotactic images.
We presented several performance results on different benchmarks data of over 30 patients in Slicer3D platform.

assistant: text: Packing region of Interest (ROI) such as cancerous regions identified in 3D Volume Data, Packing spheres inside the ROI, rotating the ROI , measures of difference in sphere packing before and after the rotation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Convergence Behaviour of Some Gradient-Based Methods on Bilinear Zero-Sum Games

Paper content:
Min-max formulations have attracted great attention in the ML community due to the rise of deep generative models and adversarial methods, and understanding the dynamics of (stochastic) gradient algorithms for solving such formulations has been a grand challenge.
As a first step, we restrict to bilinear zero-sum games and give a systematic analysis of popular gradient updates, for both simultaneous and alternating versions.
We provide exact conditions for their convergence and find the optimal parameter setup and convergence rates.
In particular, our results offer formal evidence that alternating updates converge "better" than simultaneous ones.

assistant: text: We systematically analyze the convergence behaviour of popular gradient algorithms for solving bilinear games, with both simultaneous and alternating updates.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Interpretable Robust Recommender Systems with Side Information

Paper content:
In this paper, we propose two methods, namely Trace-norm regression (TNR) and Stable Trace-norm Analysis (StaTNA), to improve performances of recommender systems with side information.
Our trace-norm regression approach extracts low-rank latent factors underlying the side information that drives user preference under different context.
Furthermore, our novel recommender framework StaTNA not only captures latent low-rank common drivers for user preferences, but also considers idiosyncratic taste for individual users.
We compare performances of TNR and StaTNA on the MovieLens datasets against state-of-the-art models, and demonstrate that StaTNA and TNR in general outperforms these methods.

assistant: text: Methodologies for recommender systems with side information based on trace-norm regularization
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Machine Learning by Two-Dimensional Hierarchical Tensor Networks: A Quantum Information Theoretic Perspective on Deep Architectures

Paper content:
The resemblance between the methods used in studying quantum-many body physics and in machine learning has drawn considerable attention.
In particular, tensor networks (TNs) and deep learning architectures bear striking similarities to the extent that TNs can be used for machine learning.
Previous results used one-dimensional TNs in image recognition, showing limited scalability and a request of high bond dimension.
In this work, we train two-dimensional hierarchical TNs to solve image recognition problems, using a training algorithm derived from the multipartite entanglement renormalization ansatz (MERA).
This approach overcomes scalability issues and implies novel mathematical connections among quantum many-body physics, quantum information theory, and machine learning.
While keeping the TN unitary in the training phase, TN states can be defined, which optimally encodes each class of the images into a quantum many-body state.
We study the quantum features of the TN states, including quantum entanglement and fidelity.
We suggest these quantities could be novel properties that characterize the image classes, as well as the machine learning tasks.
Our work could be further applied to identifying possible quantum properties of certain artificial intelligence methods.

assistant: text: This approach overcomes scalability issues and implies novel mathematical connections among quantum many-body physics, quantum information theory, and machine learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Every Sample a Task: Pushing the Limits of Heterogeneous Models with Personalized Regression

Paper content:
When data arise from multiple latent subpopulations, machine learning frameworks typically estimate parameter values independently for each sub-population.
In this paper, we propose
to overcome these limits by considering samples as tasks in a multitask learning framework.

assistant: text: We present a method to estimate collections of regression models in which each model is personalized to a single sample.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards a Deep Network Architecture for Structured Smoothness

Paper content:
We propose the Fixed Grouping Layer (FGL); a novel feedforward layer designed to incorporate the inductive bias of structured smoothness into a deep learning model.
FGL achieves this goal by connecting nodes across layers based on spatial similarity.
The use of structured smoothness, as implemented by FGL, is motivated by applications to structured spatial data, which is, in turn, motivated by domain knowledge.
The proposed model architecture outperforms conventional neural network architectures across a variety of simulated and real datasets with structured smoothness.

assistant: text: A feedforward layer to incorporate structured smoothness into a deep learning model
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title:  Sparsity Learning in Deep Neural Networks

Paper content:
The main goal of network pruning is imposing sparsity on the neural network by increasing the number of parameters with zero value in order to reduce the architecture size and the computational speedup.

assistant: text: Proposing a novel method based on the guided attention to enforce the sparisty in deep neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Self-ensembling for visual domain adaptation

Paper content:
This paper explores the use of self-ensembling for visual domain adaptation problems.
Our technique is derived from the mean teacher variant (Tarvainen et. al 2017) of temporal ensembling (Laine et al. 2017), a technique that achieved state of the art results in the area of semi-supervised learning.
We introduce a number of modifications to their approach for challenging domain adaptation scenarios and evaluate its effectiveness.
Our approach achieves state of the art results in a variety of benchmarks, including our winning entry in the VISDA-2017 visual domain adaptation challenge.
In small image benchmarks, our algorithm not only outperforms prior art, but can also achieve accuracy that is close to that of a classifier trained in a supervised fashion.

assistant: text: Self-ensembling based algorithm for visual domain adaptation, state of the art results, won VisDA-2017 image classification domain adaptation challenge.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network

Paper content:
We present a new algorithm to train a robust neural network against adversarial attacks.
Our algorithm is motivated by the following two ideas.
First, although recent work has demonstrated that fusing randomness can improve the robustness of neural networks (Liu 2017), we noticed that adding noise blindly to all the layers is not the optimal way to incorporate randomness.
Instead, we model randomness under the framework of Bayesian Neural Network (BNN) to formally learn the posterior distribution of models in a scalable way.
Second, we formulate the mini-max problem in BNN to learn the best model distribution under adversarial attacks, leading to an adversarial-trained Bayesian neural net.
Experiment results demonstrate that the proposed algorithm achieves state-of-the-art performance under strong attacks.
On CIFAR-10 with VGG network, our model leads to 14% accuracy improvement compared with adversarial training (Madry 2017) and random self-ensemble (Liu, 2017) under PGD attack with 0.035 distortion, and the gap becomes even larger on a subset of ImageNet.

assistant: text: We design an adversarial training method to Bayesian neural networks, showing a much stronger defense to white-box adversarial attacks
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
We develop end-to-end learned reconstructions for lensless mask-based cameras, including an experimental system for capturing aligned lensless and lensed images for training.
Various reconstruction methods are explored, on a scale from classic iterative approaches (based on the physical imaging model) to deep learned methods with many learned parameters.
In the middle ground, we present several variations of unrolled alternating direction method of multipliers (ADMM) with varying numbers of learned parameters.
The network structure combines knowledge of the physical imaging model with learned parameters updated from the data, which compensate for artifacts caused by physical approximations.
Our unrolled approach is 20X faster than classic methods and produces better reconstruction quality than both the classic and deep methods on our experimental system.

assistant: text: We improve the reconstruction time and quality on an experimental mask-based lensless imager using an end-to-end learning approach which incorporates knowledge of the imaging model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Stochastic AUC Maximization with Deep Neural Networks

Paper content:
Stochastic AUC maximization has garnered an increasing interest due to better fit to imbalanced data classification.
However, existing works are limited to stochastic AUC maximization with a linear predictive model, which restricts its predictive power when dealing with extremely complex data.
In this paper, we consider stochastic AUC maximization problem with a deep neural network as the predictive model.
Building on the saddle point reformulation of a surrogated loss of AUC, the problem can be cast into a {\it non-convex concave} min-max problem.
The main contribution made in this paper is to make stochastic AUC maximization more practical for deep neural networks and big data with theoretical insights as well.
In particular, we propose to explore Polyak-\L{}ojasiewicz (PL) condition that has been proved and observed in deep learning, which enables us to develop new stochastic algorithms with even faster convergence rate and more practical step size scheme.
An AdaGrad-style algorithm is also analyzed under the PL condition with adaptive convergence rate.
Our experimental results demonstrate the effectiveness of the proposed algorithms.

assistant: text: The paper designs two algorithms for the stochastic AUC maximization problem with state-of-the-art complexities when using deep neural network as predictive model, which are also verified by empirical studies.