ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Hidden incentives for self-induced distributional shift

Paper content:
Decisions made by machine learning systems have increasing influence on the world.
Yet it is common for machine learning algorithms to assume that no such influence exists.
An example is the use of the i.i.d.
assumption in online learning for applications such as content recommendation, where the (choice of) content displayed can change users' perceptions and preferences, or even drive them away, causing a shift in the distribution of users.
Generally speaking, it is possible for an algorithm to change the distribution of its own inputs.
We introduce the term self-induced distributional shift (SIDS) to describe this phenomenon.
A large body of work in reinforcement learning and causal machine learning aims to deal with distributional shift caused by deploying learning systems previously trained offline.
Our goal is similar, but distinct: we point out that changes to the learning algorithm, such as the introduction of meta-learning, can reveal hidden incentives for distributional shift (HIDS), and aim to diagnose and prevent problems associated with hidden incentives.
We design a simple  environment as a "unit test" for HIDS, as well as a content recommendation environment which allows us to disentangle different types of SIDS.
We demonstrate the potential for HIDS to cause unexpected or undesirable behavior in these environments, and propose and test a mitigation strategy.

assistant: text: The authors show how meta-learning reveals the hidden incentives for distributional shift and propose an approach based on swapping learners between environments to reduce self introduced distributional shift.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: MINE: Mutual Information Neural Estimation

Paper content:
This paper presents a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size.
MINE is  back-propable and we prove that it is strongly consistent.
We illustrate a handful of applications in which MINE is succesfully applied  to enhance the property of generative models in both unsupervised and supervised settings.
We apply our framework to estimate the information bottleneck, and apply it in tasks related to supervised classification problems.
Our results  demonstrate substantial added flexibility and improvement in these settings.

assistant: text: A scalable in sample size and dimensions mutual information estimator.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Enhancing experimental signals in single-cell RNA-sequencing data using graph signal processing

Paper content:
Single-cell RNA-sequencing (scRNA-seq) is a powerful tool for analyzing biological systems.
However, due to biological and technical noise, quantifying the effects of multiple experimental conditions presents an analytical challenge.
To overcome this challenge, we developed MELD: Manifold Enhancement of Latent Dimensions.
MELD leverages tools from graph signal processing to learn a latent dimension within the data scoring the prototypicality of each datapoint with respect to experimental or control conditions.
We call this dimension the Enhanced Experimental Signal (EES).
MELD learns the EES by filtering the noisy categorical experimental label in the graph frequency domain to recover a smooth signal with continuous values.
This method can be used to identify signature genes that vary between conditions and identify which cell types are most affected by a given perturbation.
We demonstrate the advantages of MELD analysis in two biological datasets, including T-cell activation in response to antibody-coated beads and treatment of human pancreatic islet cells with interferon gamma.

assistant: text: A novel graph signal processing framework for quantifying the effects of experimental perturbations in single cell biomedical data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Fully differentiable full-atom protein backbone generation

Paper content:
The fast generation and refinement of protein backbones would constitute a major advancement to current methodology for the design and development of de novo proteins.
In this study, we train Generative Adversarial Networks (GANs) to generate fixed-length full-atom protein backbones, with the goal of sampling from the distribution of realistic 3-D backbone fragments.
We represent protein structures by pairwise distances between all backbone atoms, and present a method for directly recovering and refining the corresponding backbone coordinates in a differentiable manner.
We show that interpolations in the latent space of the generator correspond to smooth deformations of the output backbones, and that test set structures not seen by the generator during training exist in its image.
Finally, we perform sequence design, relaxation, and ab initio folding of a subset of generated structures, and show that in some cases we can recover the generated folds after forward-folding.
Together, these results suggest a mechanism for fast protein structure refinement and folding using external energy functions.

assistant: text: A generative model for protein backbone which uses a GAN, autoencoder-like network, and refinement process, and a set of qualitative evaluations suggesting positive results.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Classification as Decoder: Trading Flexibility for Control in Multi Domain Dialogue

Paper content:
Generative seq2seq dialogue systems are trained to predict the next word in dialogues that have already occurred.
They can learn from large unlabeled conversation datasets, build a deep understanding of conversational context, and generate a wide variety of responses.
This flexibility comes at the cost of control.
Undesirable responses in the training data will be reproduced by the model at inference time, and longer generations often don’t make sense.
Instead of generating responses one word at a time, we train a classifier to choose from a predefined list of full responses.
The classifier is trained on (conversation context, response class) pairs, where each response class is a noisily labeled group of interchangeable responses.
At inference, we generate the exemplar response associated with the predicted response class.
Experts can edit and improve these exemplar responses over time without retraining the classifier or invalidating old training data.
Human evaluation of 775 unseen doctor/patient conversations shows that this tradeoff improves responses.
Only 12% of our discriminative approach’s responses are worse than the doctor’s response in the same conversational context, compared to 18% for the generative model.
A discriminative model trained without any manual labeling of response classes achieves equal performance to the generative model.

assistant: text: A way to generate responses for medical dialog using a classifier to select from expert-curated responses based on the conversation context.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: TabFact: A Large-scale Dataset for Table-based Fact Verification

Paper content:
The problem of verifying whether a textual hypothesis holds based on the given evidence, also known as fact verification, plays an important role in the study of natural language understanding and semantic representation.
However, existing studies are mainly restricted to dealing with unstructured evidence (e.g., natural language sentences and documents, news, etc), while verification under structured evidence, such as tables, graphs, and databases, remains unexplored.
This paper specifically aims to study the fact verification given semi-structured data as evidence.
To this end, we construct a large-scale dataset called TabFact with 16k Wikipedia tables as the evidence for 118k human-annotated natural language statements, which are labeled as either ENTAILED or REFUTED.
TabFact is challenging since it involves both soft linguistic reasoning and hard symbolic reasoning.
To address these reasoning challenges, we design two different models: Table-BERT and Latent Program Algorithm (LPA).
Table-BERT leverages the state-of-the-art pre-trained language model to encode the linearized tables and statements into continuous vectors for verification.
LPA parses statements into LISP-like programs and executes them against the tables to obtain the returned binary value for verification.
Both methods achieve similar accuracy but still lag far behind human performance.
We also perform a comprehensive analysis to demonstrate great future opportunities.

assistant: text: This paper proposes a new dataset for table-based fact verification and introduces methods for the task.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Linearizing Visual Processes with Deep Generative Models

Paper content:
This work studies the problem of modeling non-linear visual processes by leveraging deep generative architectures for learning linear, Gaussian models of observed sequences.
We propose a joint learning framework, combining a multivariate autoregressive model and deep convolutional generative networks.
After justification of theoretical assumptions of inearization, we propose an architecture that allows Variational Autoencoders and Generative Adversarial Networks to simultaneously learn the non-linear observation as well as the linear state-transition model from a sequence of observed frames.
Finally, we demonstrate our approach on conceptual toy examples and dynamic textures.

assistant: text: We model non-linear visual processes as autoregressive noise via generative deep learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Fast is better than free: Revisiting adversarial training

Paper content:
Adversarial training, a method for learning robust deep networks, is typically assumed to be more expensive than traditional training due to the necessity of constructing adversarial examples via a first-order method like projected gradient decent (PGD).
In this paper, we make the surprising discovery that it is possible to train empirically robust models using a much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice.
Specifically, we show that adversarial training with the fast gradient sign method (FGSM), when combined with random initialization, is as effective as PGD-based training but has significantly lower cost.
Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy at epsilon=8/255 in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at epsilon=2/255 in 12 hours, in comparison to past work based on ``free'' adversarial training which took 10 and 50 hours to reach the same respective thresholds.

assistant: text: The main claim of this paper is that a simple strategy of randomization plus fast gradient sign method (FGSM) adversarial training yields robust neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Neural Arithmetic Unit by reusing many small pre-trained networks

Paper content:
We propose a solution for evaluation of mathematical expression.
However, instead of designing a single end-to-end model we propose a Lego bricks style architecture.
In this architecture instead of training a complex end-to-end neural network, many small networks can be trained independently each accomplishing one specific operation and acting a single lego brick.
More difficult or complex task can then be solved using a combination of these smaller network.
In this work we first identify 8 fundamental operations that are commonly used to solve arithmetic operations (such as 1 digit multiplication, addition, subtraction, sign calculator etc).
These fundamental operations are then learned using simple feed forward neural networks.
We then shows that different operations can be designed simply by reusing these smaller networks.
As an example we reuse these smaller networks to develop larger and a more complex network to solve n-digit multiplication, n-digit division, and cross product.
This bottom-up strategy not only introduces reusability, we also show that it allows to generalize for computations involving n-digits and we show results for up to 7 digit numbers.
Unlike existing methods, our solution also generalizes for both positive as well as negative numbers.

assistant: text: This paper proposes to use neural networks to evaluate the mathematical expressions by designing 8 small building blocks for 8 fundamental operations, e.g., addition, subtraction, etc and then designing multi-digit multiplication and division using these small blocks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputation

Paper content:
Many real-world applications involve multivariate, geo-tagged time series data: at each location, multiple sensors record corresponding measurements.
For example, air quality monitoring system records PM2.5, CO, etc.
The resulting time-series data often has missing values due to device outages or communication errors.
In order to impute the missing values, state-of-the-art methods are built on Recurrent Neural Networks (RNN), which process each time stamp sequentially, prohibiting the direct modeling of the relationship between distant time stamps.
Recently, the self-attention mechanism has been proposed for sequence modeling tasks such as machine translation, significantly outperforming RNN because the relationship between each two time stamps can be modeled explicitly.
In this paper, we are the first to adapt the self-attention mechanism for multivariate, geo-tagged time series data.
In order to jointly capture the self-attention across different dimensions (i.e. time, location and sensor measurements) while keep the size of attention maps reasonable, we propose a novel approach called Cross-Dimensional Self-Attention (CDSA) to process each dimension sequentially, yet in an order-independent manner.
On three real-world datasets, including one our newly collected NYC-traffic dataset, extensive experiments demonstrate the superiority of our approach compared to state-of-the-art methods for both imputation and forecasting tasks.

assistant: text: A novel self-attention mechanism for multivariate, geo-tagged time series imputation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Balanced and Deterministic Weight-sharing Helps Network Performance

Paper content:
Weight-sharing plays a significant role in the success of many deep neural networks, by increasing memory efficiency and incorporating useful inductive priors about the problem into the network.
But understanding how weight-sharing can be used effectively in general is a topic that has not been studied extensively.
Chen et al. (2015) proposed HashedNets, which augments a multi-layer perceptron with a hash table, as a method for neural network compression.
We generalize this method into a framework (ArbNets) that allows for efficient arbitrary weight-sharing, and use it to study the role of weight-sharing in neural networks.
We show that common neural networks can be expressed as ArbNets with different hash functions.
We also present two novel hash functions, the Dirichlet hash and the Neighborhood hash, and use them to demonstrate experimentally that balanced and deterministic weight-sharing helps with the performance of a neural network.

assistant: text: Studied the role of weight sharing in neural networks using hash functions, found that a balanced and deterministic hash function helps network performance.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Policy Message Passing: A New Algorithm for Probabilistic Graph Inference

Paper content:
A general graph-structured neural network architecture operates on graphs through two core components: (1) complex enough message functions; (2) a fixed information aggregation process.
In this paper, we present the Policy Message Passing algorithm, which takes a probabilistic perspective and reformulates the whole information aggregation as stochastic sequential processes.
The algorithm works on a much larger search space, utilizes reasoning history to perform inference, and is robust to noisy edges.
We apply our algorithm to multiple complex graph reasoning and prediction tasks and show that our algorithm consistently outperforms state-of-the-art graph-structured models by a significant margin.

assistant: text: This paper introduces policy message passing, a graph neural network with an inference mechanism that assigns messages to edges in a recurrent fashion, indicating competitive performance on visual reasoning tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Mixed-curvature Variational Autoencoders

Paper content:
It has been shown that using geometric spaces with non-zero curvature instead of plain Euclidean spaces with zero curvature improves performance on a range of Machine Learning tasks for learning representations.
Recent work has leveraged these geometries to learn latent variable models like Variational Autoencoders (VAEs) in spherical and hyperbolic spaces with constant curvature.
While these approaches work well on particular kinds of data that they were designed for e.g.~tree-like data for a hyperbolic VAE, there exists no generic approach unifying all three models.
We develop a Mixed-curvature Variational Autoencoder, an efficient way to train a VAE whose latent space is a product of constant curvature Riemannian manifolds, where the per-component curvature can be learned.
This generalizes the Euclidean VAE to curved latent spaces, as the model essentially reduces to the Euclidean VAE if curvatures of all latent space components go to 0.

assistant: text: This paper is about developing VAEs in non-Euclidean spaces.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Revisiting Auxiliary Latent Variables in Generative Models

Paper content:
Extending models with auxiliary latent variables is a well-known technique to in-crease model expressivity.
Bachman & Precup (2015); Naesseth et al. (2018); Cremer et al. (2017); Domke & Sheldon (2018) show that Importance Weighted Autoencoders (IWAE) (Burda et al., 2015) can be viewed as extending the variational family with auxiliary latent variables.
Similarly, we show that this view encompasses many of the recent developments in variational bounds (Maddisonet al., 2017; Naesseth et al., 2018; Le et al., 2017; Yin & Zhou, 2018; Molchanovet al., 2018; Sobolev & Vetrov, 2018).
The success of enriching the variational family with auxiliary latent variables motivates applying the same techniques to the generative model.
We develop a generative model analogous to the IWAE bound and empirically show that it outperforms the recently proposed Learned Accept/Reject Sampling algorithm (Bauer & Mnih, 2018), while being substantially easier to implement.
Furthermore, we show that this generative process provides new insights on ranking Noise Contrastive Estimation (Jozefowicz et al.,2016; Ma & Collins, 2018) and Contrastive Predictive Coding (Oord et al., 2018).

assistant: text: Proposes a different view on improving variational bounds with auxiliary latent variable models and explores the use of those models in the generative model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Challenges of Explaining Control

Paper content:
Reinforcement learning and evolutionary algorithms can be used to create sophisticated control solutions.
Unfortunately explaining how these solutions work can be difficult to due to their "black box" nature.
In addition, the time-extended nature of control algorithms often prevent direct applications of explainability techniques used for standard supervised learning algorithms.
This paper attempts to address explainability of blackbox control algorithms through six different techniques:
1) Bayesian rule lists,
2) Function analysis,
3) Single time step integrated gradients,
4) Grammar-based decision trees,
5) Sensitivity analysis combined with temporal modeling with LSTMs, and
6) Explanation templates.
These techniques are tested on a simple 2d domain, where a simulated rover attempts to navigate through obstacles to reach a goal.
For control, this rover uses an evolved multi-layer perception that maps an 8d field of obstacle and goal sensors to an action determining where it should go in the next time step.
Results show that some simple insights in explaining the neural network are possible, but that good explanations are difficult.

assistant: text: Describes a series of explainability techniques applied to a simple neural network controller used for navigation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring in Data

Paper content:
Equivariance is a nice property to have as it produces much more parameter efficient neural architectures and preserves the structure of the input through the feature mapping.
Even though some combinations of transformations might never appear (e.g. an upright face with a horizontal nose), current equivariant architectures consider the set of all possible transformations in a transformation group when learning feature representations.
Contrarily, the human visual system is able to attend to the set of relevant transformations occurring in the environment and utilizes this information to assist and improve object recognition.
Based on this observation, we modify conventional equivariant feature mappings such that they are able to attend to the set of co-occurring transformations in data and generalize this notion to act on groups consisting of multiple symmetries.
We show that our proposed co-attentive equivariant neural networks consistently outperform conventional rotation equivariant and rotation & reflection equivariant neural networks on rotated MNIST and CIFAR-10.

assistant: text: The authors propose a self-attention mechanism for rotation-equivariant neural nets that improves classification performance over regular rotation-equivariant nets.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On the Effectiveness of Minimal Context Selection for Robust Question Answering

Paper content:
Machine learning models for question-answering (QA), where given a question and a passage, the learner must select some span in the passage as an answer, are known to be brittle.
By inserting a single nuisance sentence into the passage, an adversary can fool the model into selecting the wrong span.
A promising new approach for QA decomposes the task into two stages:
(i) select relevant sentences from the passage; and
(ii) select a span among those sentences.
Intuitively, if the sentence selector excludes the offending sentence, then the downstream span selector will be robust.
While recent work has hinted at the potential robustness of two-stage QA, these methods have never, to our knowledge, been explicitly combined with adversarial training.
This paper offers a thorough empirical investigation of adversarial robustness, demonstrating that although the two-stage approach lags behind single-stage span selection, adversarial training improves its performance significantly, leading to an improvement of over 22 points in F1 score over the adversarially-trained single-stage model.

assistant: text: A two-stage approach consisting of sentence selection followed by span selection can be made more robust to adversarial attacks in comparison to a single-stage model trained on full context.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Word2net: Deep Representations of Language

Paper content:
Word embeddings extract semantic features of words from large datasets of text.
Most embedding methods rely on a log-bilinear model to predict the occurrence
of a word in a context of other words.
Here we propose word2net, a method that
replaces their linear parametrization with neural networks.
For each term in the
vocabulary, word2net posits a neural network that takes the context as input and
outputs a probability of occurrence.
Further, word2net can use the hierarchical
organization of its word networks to incorporate additional meta-data, such as
syntactic features, into the embedding model.
For example, we show how to share
parameters across word networks to develop an embedding model that includes
part-of-speech information.
We study word2net with two datasets, a collection
of Wikipedia articles and a corpus of U.S. Senate speeches.
Quantitatively, we
found that word2net outperforms popular embedding methods on predicting held-
out words and that sharing parameters based on part of speech further boosts
performance.
Qualitatively, word2net learns interpretable semantic representations
and, compared to vector-based methods, better incorporates syntactic information.

assistant: text: This paper extends SGNS with an architectural change from a bag-of-words model to a feedforward model, and contributes a new form of regularization by tying a subset of layers between different associated networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: I love your chain mail! Making knights smile in a fantasy game world

Paper content:
Dialogue research tends to distinguish between chit-chat and goal-oriented tasks.
While the former is arguably more naturalistic and has a wider use of language, the latter has clearer metrics and a more straightforward learning signal.
Humans effortlessly combine the two, and engage in chit-chat for example with the goal of exchanging information or eliciting a specific response.
Here, we bridge the divide between these two domains in the setting of a rich multi-player text-based fantasy environment where agents and humans engage in both actions and dialogue.
Specifically, we train a goal-oriented model with reinforcement learning via self-play against an imitation-learned chit-chat model with two new approaches: the policy either learns to pick a topic or learns to pick an utterance given the top-k utterances.
We show that both models outperform a strong inverse model baseline and can converse naturally with their dialogue partner in order to achieve goals.

assistant: text: This paper explores the goal-oriented dialogue setting with reinforcement learning in a Fantasy Text Adventure Game and observes that the RL approaches outperform supervised learning models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Automated Science Scheduling for the ECOSTRESS Mission

Paper content:
We describe the use of an automated scheduling system for observation policy design and to schedule operations of the NASA (National Aeronautics and Space Administration) ECOSystem Spaceborne Thermal Radiometer Experiment on Space Station (ECOSTRESS).
We describe the adaptation of the Compressed Large-scale Activity Scheduler and Planner (CLASP) scheduling system to the ECOSTRESS scheduling problem, highlighting multiple use cases for automated scheduling and several challenges for the scheduling technology: handling long-term campaigns with changing information, Mass Storage Unit Ring Buffer operations challenges, and orbit uncertainty.
The described scheduling system has been used for operations of the ECOSTRESS instrument since its nominal operations start July 2018 and is expected to operate until mission end in Summer 2019.

assistant: text: We describe the use of an automated scheduling system for observation policy design and to schedule operations of NASA's ECOSTRESS mission.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Incorporating Horizontal Connections in Convolution by Spatial Shuffling

Paper content:
Convolutional Neural Networks (CNNs) are composed of multiple convolution layers and show elegant performance in vision tasks.
The design of the regular convolution is based on the Receptive Field (RF) where the information within a specific region is processed.
In the view of the regular convolution's RF, the outputs of neurons in lower layers with smaller RF are bundled to create neurons in higher layers with larger RF.
As a result, the neurons in high layers are able to capture the global context even though the neurons in low layers only see the local information.
However, in lower layers of the biological brain, the information outside of the RF changes the properties of neurons.
In this work, we extend the regular convolution and propose spatially shuffled convolution (ss convolution).
In ss convolution, the regular convolution is able to use the information outside of its RF by spatial shuffling which is a simple and lightweight operation.
We perform experiments on CIFAR-10 and ImageNet-1k dataset, and show that ss convolution improves the classification performance across various CNNs.

assistant: text: The authors proposed a shuffle strategy for convolution layers in convolution layers in convolutional neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient Receptive Field Learning by Dynamic Gaussian Structure

Paper content:
The visual world is vast and varied, but its variations divide into structured and unstructured factors.
Structured factors, such as scale and orientation, admit clear theories and efficient representation design.
Unstructured factors, such as what it is that makes a cat look like a cat, are too complicated to model analytically, and so require free-form representation learning.
We compose structured Gaussian filters and free-form filters, optimized end-to-end, to factorize the representation for efficient yet general learning.
Our experiments on dynamic structure, in which the structured filters vary with the input, equal the accuracy of dynamic inference with more degrees of freedom while improving efficiency.
(Please see https://arxiv.org/abs/1904.11487 for the full edition.)

assistant: text: Dynamic receptive fields with spatial Gaussian structure are accurate and efficient.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Structured Pruning for Efficient ConvNets via Incremental Regularization

Paper content:
Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance loss.
Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fact that the expressiveness of CNNs is fragile and needs a more gentle way of regularization for the networks to adapt during pruning.
To solve this problem, we propose a new regularization-based pruning method (named IncReg) to incrementally assign different regularization factors to different weight groups based on their relative importance, whose effectiveness is proved on popular CNNs compared with state-of-the-art methods.

assistant: text: we propose a new regularization-based pruning method (named IncReg) to incrementally assign different regularization factors to different weight groups based on their relative importance.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Joint autoencoders: a flexible meta-learning framework

Paper content:
The incorporation of prior knowledge into learning is essential in achieving good performance based on small noisy samples.
Such knowledge is often incorporated through the availability of related data arising from domains and tasks similar to the one of current interest.
Ideally one would like to allow both the data for the current task and for previous related tasks to self-organize the learning system in such a way that commonalities and differences between the tasks are learned in a data-driven fashion.
We develop a framework for learning multiple tasks simultaneously, based on sharing features that are common to all tasks, achieved through the use of a modular deep feedforward neural network consisting of shared branches, dealing with the common features of all tasks, and private branches, learning the specific unique aspects of each task.
Once an appropriate weight sharing architecture has been established, learning takes place through standard algorithms for feedforward networks, e.g., stochastic gradient descent and its variations.
The method deals with meta-learning (such as domain adaptation, transfer and multi-task learning) in a unified fashion, and can easily deal with data arising from different types of sources.
Numerical experiments demonstrate the effectiveness of learning in domain adaptation and transfer learning setups, and provide evidence for the flexible and task-oriented representations arising in the network.

assistant: text: This paper proposes a model for allowing deep neural network architectures to share parameters across different datasets, and applies it to transfer learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: PDE-Net: Learning PDEs from Data

Paper content:
Partial differential equations (PDEs)  play a prominent role in many disciplines such as applied mathematics, physics, chemistry, material science, computer science, etc.
PDEs are commonly derived based on physical laws or empirical observations.
However, the governing equations for many complex systems in modern applications are still not fully known.
With the rapid development of sensors, computational power, and data storage in the past decade, huge quantities of data can be easily collected and efficiently stored.
Such vast quantity of data offers new opportunities for data-driven discovery of hidden physical laws.
Inspired by the latest development of neural network designs in deep learning, we propose a new feed-forward deep network, called PDE-Net, to fulfill two objectives at the same time: to accurately predict dynamics of complex systems and to uncover the underlying hidden PDE models.
The basic idea of the proposed PDE-Net is to learn differential operators by learning convolution kernels (filters), and apply neural networks or other machine learning methods to approximate the unknown nonlinear responses.
Comparing with existing approaches, which either assume the form of the nonlinear response is known or fix certain finite difference approximations of differential operators, our approach has the most flexibility by learning both differential operators and the nonlinear responses.
A special feature of the proposed PDE-Net is that all filters are properly constrained, which enables us to easily identify the governing PDE models while still maintaining the expressive and predictive power of the network.
These constrains are carefully designed by fully exploiting the relation between the orders of differential operators and the orders of sum rules of filters (an important concept originated from wavelet theory).
We also discuss relations of the PDE-Net with some existing networks in computer vision such as Network-In-Network (NIN) and Residual Neural Network (ResNet).
Numerical experiments show that the PDE-Net has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.

assistant: text: This paper addresses complex dynamical systems modelling through nonparametric Partial Differential Equations using neural architectures, with the most important idea of the papier (PDE-net) to learn both differential operators and the function that governs the PDE.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Discrete flow posteriors for variational inference in discrete dynamical systems

Paper content:
Each training step for a variational autoencoder (VAE) requires us to sample from the approximate posterior, so we usually choose simple (e.g. factorised) approximate posteriors in which sampling is an efficient computation that fully exploits GPU parallelism.
However, such simple approximate posteriors are often insufficient, as they eliminate statistical dependencies in the posterior.
While it is possible to use normalizing flow approximate posteriors for continuous latents, there is nothing analogous for discrete latents.
The most natural approach to model discrete dependencies is an autoregressive distribution, but sampling from such distributions is inherently sequential and thus slow.
We develop a fast, parallel sampling procedure for autoregressive distributions based on fixed-point iterations which enables efficient and accurate variational inference in discrete state-space models.
To optimize the variational bound, we considered two ways to evaluate probabilities: inserting the relaxed samples directly into the pmf for the discrete distribution, or converting to continuous logistic latent variables and interpreting the K-step fixed-point iterations as a normalizing flow.
We found that converting to continuous latent variables gave considerable additional scope for mismatch between the true and approximate posteriors, which resulted in biased inferences, we thus used the former approach.
We tested our approach on the neuroscience problem of inferring discrete spiking activity from noisy calcium-imaging data, and found that it gave accurate connectivity estimates in an order of magnitude less time.

assistant: text: We give a fast normalising-flow like sampling procedure for discrete latent variable models.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Classification of Building Noise Type/Position via Supervised Learning

Paper content:
This paper presents noise type/position classification of various impact noises generated in a building which is a serious conflict issue in apartment complexes.
For this study, a collection of floor impact noise dataset is recorded with a single microphone.
Noise types/positions are selected based on a report by the Floor Management Center under Korea Environmental Corporation.
Using a convolutional neural networks based classifier, the impact noise signals converted to log-scaled Mel-spectrograms are classified into noise types or positions.
Also, our model is evaluated on a standard environmental sound dataset ESC-50 to show extensibility on environmental sound classification.

assistant: text: This paper presents noise type/position classification of various impact noises generated in a building which is a serious conflict issue in apartment complexes
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Semi-supervised 3D Face Reconstruction with Nonlinear Disentangled Representations

Paper content:
Recovering 3D geometry shape, albedo and lighting from a single image has wide applications in many areas, which is also a typical ill-posed problem.
In order to eliminate the ambiguity, face prior knowledge like linear 3D morphable models (3DMM) learned from limited scan data are often adopted to the reconstruction process.
However, methods based on linear parametric models cannot generalize well for facial images in the wild with various ages, ethnicity, expressions, poses, and lightings.
Recent methods aim to learn a nonlinear parametric model using convolutional neural networks (CNN) to regress the face shape and texture directly.
However, the models were only trained on a dataset that is generated from a linear 3DMM.
Moreover, the identity and expression representations are entangled in these models, which hurdles many facial editing applications.
In this paper, we train our model with adversarial loss in a semi-supervised manner on hybrid batches of unlabeled and labeled face images to exploit the value of large amounts of unlabeled face images from unconstrained photo collections.
A novel center loss is introduced to make sure that different facial images from the same person have the same identity shape and albedo.
Besides, our proposed model disentangles identity, expression, pose, and lighting representations, which improves the overall reconstruction performance and facilitates facial editing applications, e.g., expression transfer.
Comprehensive experiments demonstrate that our model produces high-quality reconstruction compared to state-of-the-art methods and is robust to various expression, pose, and lighting conditions.

assistant: text: We train our face reconstruction model with adversarial loss in semi-supervised manner on hybrid batches of unlabeled and labeled face images to exploit the value of large amounts of unlabeled face images from unconstrained photo collections.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Progressive Knowledge Distillation For Generative Modeling

Paper content:
While modern  generative  models are able to synthesize high-fidelity, visually appealing images, successfully generating examples that are useful for recognition tasks remains an elusive goal.
To this end, our key insight is that the examples should be synthesized to recover classifier decision boundaries that would be learned from a large amount of real examples.
More concretely, we treat a classifier trained on synthetic examples as ''student'' and a classifier trained on real examples as ''teacher''.
By introducing knowledge distillation into a meta-learning framework, we encourage the generative model to produce examples in a way that enables the student classifier to mimic the behavior of the teacher.
To mitigate the potential gap between student and teacher classifiers, we further propose to distill the knowledge in a progressive manner, either by gradually strengthening the teacher or weakening the student.
We demonstrate the use of our model-agnostic distillation approach to deal with data scarcity, significantly improving few-shot learning performance on miniImageNet and ImageNet1K benchmarks.

assistant: text: This paper introduces progressive knowledge distillation for learning generative models that are recognition task oriented
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Sparse Coding with Gated Learned ISTA

Paper content:
In this paper, we study the learned iterative shrinkage thresholding algorithm (LISTA) for solving sparse coding problems.
Following assumptions made by prior works, we first discover that the code components in its estimations may be lower than expected, i.e., require gains, and to address this problem, a gated mechanism amenable to theoretical analysis is then introduced.
Specific design of the gates is inspired by convergence analyses of the mechanism and hence its effectiveness can be formally guaranteed.
In addition to the gain gates, we further introduce overshoot gates for compensating insufficient step size in LISTA.
Extensive empirical results confirm our theoretical findings and verify the effectiveness of our method.

assistant: text: Proposes extensions to LISTA which address underestimation by introducing "gain gates" and including momentum with "overshoot gates", showing improved convergence rates.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: LSH-SAMPLING BREAKS THE COMPUTATIONAL CHICKEN-AND-EGG LOOP IN ADAPTIVE STOCHASTIC GRADIENT ESTIMATION

Paper content:
Stochastic Gradient Descent or SGD is the most popular optimization algorithm for large-scale problems.
SGD estimates the gradient by uniform sampling with sample size one.
There have been several other works that suggest faster epoch wise convergence by using weighted non-uniform sampling for better gradient estimates.
Unfortunately, the per-iteration cost of maintaining this adaptive distribution for gradient estimation is more than calculating the full gradient.
As a result, the false impression of faster convergence in iterations leads to slower convergence in time, which we call a chicken-and-egg loop.
In this paper, we break this barrier by providing the first demonstration of a sampling scheme, which leads to superior gradient estimation, while keeping the sampling cost per iteration similar to that of the uniform sampling.
Such an algorithm is possible due to the sampling view of Locality Sensitive Hashing (LSH), which came to light recently.
As a consequence of superior and fast estimation, we reduce the running time of all existing gradient descent algorithms.
We demonstrate the benefits of our proposal on both SGD and AdaGrad.
In this paper, we proposed a novel LSH-based sampler with a reduction to the gradient estimation variance.
We achieved it by sampling with probability proportional to the L 2 norm of the instances gradients leading to an optimal distribution that minimizes the variance of estimation.
More remarkably, LSD is as computationally efficient as SGD but achieves faster convergence not only epoch wise but also time wise.Peilin Zhao and Tong Zhang.
Stochastic optimization with importance sampling for regularized loss minimization.
In Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pp. 1-9, 2015.A EPOCH PLOTS AND PROOFS Theorem 3.
Let S be the bucket that sample x m is chosen from in Algorithm 2.
Let p m be the sampling probability associated with sample x m .
Suppose we query a sample with ✓ t .
Then we have an unbiased estimator of the full gradient: DISPLAYFORM0 Proof.
DISPLAYFORM1 Theorem 4.
The Trace of the covariance of our estimator is: DISPLAYFORM2 Proof.

assistant: text: Authors propose sampling stochastic gradients from a monotonic function proportional to gradient magnitudes by using LSH.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Spectral Capsule Networks

Paper content:
In search for more accurate predictive models, we customize capsule networks for the learning to diagnose problem.
We also propose Spectral Capsule Networks, a novel variation of capsule networks, that converge faster than capsule network with EM routing.
Spectral capsule networks  consist of spatial coincidence filters that detect entities based on the alignment of extracted features on a one-dimensional linear subspace.
Experiments on a public benchmark learning to diagnose dataset not only shows the success of capsule networks on this task, but also confirm the faster convergence of the spectral capsule networks.

assistant: text: Presents a variant of capsule networks that instead of using EM routing employs a linear subspace spanned by the dominant eigenvector on the weighted votes matrix from the previous capsule.