ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Open-Set Domain Adaptation with Category-Agnostic Clusters

Paper content:
Unsupervised domain adaptation has received significant attention in recent years.
Most of existing works tackle the closed-set scenario, assuming that the source and target domains share the exactly same categories.
In practice, nevertheless, a target domain often contains samples of classes unseen in source domain (i.e., unknown class).
The extension of domain adaptation from closed-set to such open-set situation is not trivial since the target samples in unknown class are not expected to align with the source.
In this paper, we address this problem by augmenting the state-of-the-art domain adaptation technique, Self-Ensembling, with category-agnostic clusters in target domain.
Specifically, we present Self-Ensembling with Category-agnostic Clusters (SE-CC) --- a novel architecture that steers domain adaptation with the additional guidance of category-agnostic clusters that are specific to target domain.
These clustering information provides domain-specific visual cues, facilitating the generalization of Self-Ensembling for both closed-set and open-set scenarios.
Technically, clustering is firstly performed over all the unlabeled target samples to obtain the category-agnostic clusters, which reveal the underlying data space structure peculiar to target domain.
A clustering branch is capitalized on to ensure that the learnt representation preserves such underlying structure by matching the estimated assignment distribution over clusters to the inherent cluster distribution for each target sample.
Furthermore, SE-CC enhances the learnt representation with mutual information maximization.
Extensive experiments are conducted on Office and VisDA datasets for both open-set and closed-set domain adaptation, and superior results are reported when comparing to the state-of-the-art approaches.

assistant: text: We present a new design, i.e., Self-Ensembling with Category-agnostic Clusters, for both closed-set and open-set domain adaptation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Riemannian Stochastic Gradient Descent for Tensor-Train Recurrent Neural Networks

Paper content:
The Tensor-Train factorization (TTF) is an efficient way to compress large weight matrices of fully-connected layers and recurrent layers in recurrent neural networks (RNNs).
However, high Tensor-Train ranks for all the core tensors of parameters need to be element-wise fixed, which results in an unnecessary redundancy of model parameters.
This work applies Riemannian stochastic gradient descent (RSGD) to train core tensors of parameters in the Riemannian Manifold before finding vectors of lower Tensor-Train ranks for parameters.
The paper first presents the RSGD algorithm with a convergence analysis and then tests it on more advanced Tensor-Train RNNs such as bi-directional GRU/LSTM and Encoder-Decoder RNNs with a Tensor-Train attention model.
The experiments on digit recognition and machine translation tasks suggest the effectiveness of the RSGD algorithm for Tensor-Train RNNs.

assistant: text: Applying the Riemannian SGD (RSGD) algorithm for training Tensor-Train RNNs to further reduce model parameters.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Mitigating Posterior Collapse in Strongly Conditioned Variational Autoencoders

Paper content:
Training conditional generative latent-variable models is challenging in scenarios where the conditioning signal is very strong and the decoder is expressive enough to generate a plausible output given only the condition; the generative model tends to ignore the latent variable, suffering from posterior collapse.
We find, and empirically show, that one of the major reasons behind posterior collapse is rooted in the way that generative models are conditioned, i.e., through concatenation of the latent variable and the condition
. To mitigate this problem, we propose to explicitly make the latent variables depend on the condition by unifying the conditioning and latent variable sampling, thus coupling them so as to prevent the model from discarding the root of variations
. To achieve this, we develop a conditional Variational Autoencoder architecture that learns a distribution not only of the latent variables, but also of the condition, the latter acting as prior on the former
. Our experiments on the challenging tasks of conditional human motion prediction and image captioning demonstrate the effectiveness of our approach at avoiding posterior collapse
. Video results of our approach are anonymously provided in http://bit.ly/iclr2020

assistant: text: We propose a conditional variational autoencoder framework that mitigates the posterior collapse in scenarios where the conditioning signal strong enough for an expressive decoder to generate a plausible output from it.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention

Paper content:
A point cloud is an agile 3D representation, efficiently modeling an object's surface geometry.
However, these surface-centric properties also pose challenges on designing tools to recognize and synthesize point clouds.
This work presents a novel autoregressive model, PointGrow, which generates realistic point cloud samples from scratch or conditioned from given semantic contexts.
Our model operates recurrently, with each point sampled according to a conditional distribution given its previously-generated points.
Since point cloud object shapes are typically encoded by long-range interpoint dependencies, we augment our model with dedicated self-attention modules to capture these relations.
Extensive evaluation demonstrates that PointGrow achieves satisfying performance on both unconditional and conditional point cloud generation tasks, with respect to fidelity, diversity and semantic preservation.
Further, conditional PointGrow learns a smooth manifold of given images where 3D shape interpolation and arithmetic calculation can be performed inside.

assistant: text: An autoregressive deep learning model for generating diverse point clouds.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Neural Arithmetic Unit by reusing many small pre-trained networks

Paper content:
We propose a solution for evaluation of mathematical expression.
However, instead of designing a single end-to-end model we propose a Lego bricks style architecture.
In this architecture instead of training a complex end-to-end neural network, many small networks can be trained independently each accomplishing one specific operation and acting a single lego brick.
More difficult or complex task can then be solved using a combination of these smaller network.
In this work we first identify 8 fundamental operations that are commonly used to solve arithmetic operations (such as 1 digit multiplication, addition, subtraction, sign calculator etc).
These fundamental operations are then learned using simple feed forward neural networks.
We then shows that different operations can be designed simply by reusing these smaller networks.
As an example we reuse these smaller networks to develop larger and a more complex network to solve n-digit multiplication, n-digit division, and cross product.
This bottom-up strategy not only introduces reusability, we also show that it allows to generalize for computations involving n-digits and we show results for up to 7 digit numbers.
Unlike existing methods, our solution also generalizes for both positive as well as negative numbers.

assistant: text: We train many small networks each for a specific operation, these are then combined to perform complex operations
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Déjà Vu: An Empirical Evaluation of the Memorization Properties of Convnets

Paper content:
Convolutional neural networks memorize part of their training data, which is why strategies such as data augmentation and drop-out are employed to mitigate over- fitting.
This paper considers the related question of “membership inference”, where the goal is to determine if an image was used during training.
We con- sider membership tests over either ensembles of samples or individual samples.
First, we show how to detect if a dataset was used to train a model, and in particular whether some validation images were used at train time.
Then, we introduce a new approach to infer membership when a few of the top layers are not available or have been fine-tuned, and show that lower layers still carry information about the training samples.
To support our findings, we conduct large-scale experiments on Imagenet and subsets of YFCC-100M with modern architectures such as VGG and Resnet.

assistant: text: Illuminates the generalization/memorization properties of large and deep ConvNets and tries to develop procedures related to identifying whether an input to a trained ConvNet has actually been used to train the network.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Lipschitz constant estimation of Neural Networks via sparse polynomial optimization

Paper content:
We introduce LiPopt, a polynomial optimization framework for computing increasingly tighter upper bound on the Lipschitz constant of neural networks.
The underlying optimization problems boil down to either linear (LP) or semidefinite (SDP) programming.
We show how to use the sparse connectivity of a network, to significantly reduce the complexity of computation.
This is specially useful for convolutional as well as pruned neural networks.
We conduct experiments on networks with random weights as well as networks trained on MNIST, showing that in the particular case of the $\ell_\infty$-Lipschitz constant, our approach yields superior estimates as compared to other baselines available in the literature.

assistant: text: The authors study the problem of estimating the Lipschitz constant of a deep neural network with ELO activation function, formulating it as a polynomial optimisation problem.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Scalable Neural Learning for Verifiable Consistency with Temporal Specifications

Paper content:
Formal verification of machine learning models has attracted attention recently, and significant progress has been made on proving simple properties like robustness to small perturbations of the input features.
In this context, it has also been observed that folding the verification procedure into training makes it easier to train verifiably robust models.
In this paper, we extend the applicability of verified training by extending it to (1) recurrent neural network architectures and (2) complex specifications that go beyond simple adversarial robustness, particularly specifications that capture temporal properties like requiring that a robot periodically visits a charging station or that a language model always produces sentences of bounded length.
Experiments show that while models trained using standard training often violate desired specifications, our verified training method produces models that both perform well (in terms of test error or reward) and can be shown to be provably consistent with specifications.

assistant: text: Neural Network Verification for Temporal Properties and Sequence Generation Models
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Augmenting Self-attention with Persistent Memory

Paper content:
Transformer networks have lead to important progress in language modeling and machine translation.
These models include two consecutive modules, a feed-forward layer and a self-attention layer.
The latter allows the network to capture long term dependencies and are often regarded as the key ingredient in the success of Transformers.
Building upon this intuition, we propose a new model that solely consists of attention layers.
More precisely, we augment the self-attention layers with persistent memory vectors that play a similar role as the feed-forward layer.
Thanks to these vectors, we can remove the feed-forward layer without degrading the performance of a transformer.
Our evaluation shows the benefits brought by our model on standard character and word level language modeling benchmarks.

assistant: text: A novel attention layer that combines self-attention and feed-forward sublayers of Transformer networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adaptive Quantization of Neural Networks

Paper content:
Despite the state-of-the-art accuracy of Deep Neural Networks (DNN) in various classification problems, their deployment onto resource constrained edge computing devices remains challenging due to their large size and complexity.
Several recent studies have reported remarkable results in reducing this complexity through quantization of DNN models.
However, these studies usually do not consider the changes in the loss function when performing quantization, nor do they take the different importances of DNN model parameters to the accuracy into account.
We address these issues in this paper by proposing a new method, called adaptive quantization, which simplifies a trained DNN model by finding a unique, optimal precision for each network parameter such that the increase in loss is minimized.
The optimization problem at the core of this method iteratively uses the loss function gradient to determine an error margin for each parameter and assigns it a precision accordingly.
Since this problem uses linear functions, it is computationally cheap and, as we will show, has a closed-form approximate solution.
Experiments on MNIST, CIFAR, and SVHN datasets showed that the proposed method can achieve near or better than state-of-the-art reduction in model size with similar error rates.
Furthermore, it can achieve compressions close to floating-point model compression methods without loss of accuracy.

assistant: text: The paper proposes a technique for quantizing the weights of a neural network with bit-depth/precision varying on a per-parameter basis.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables

Paper content:
Deep reinforcement learning algorithms require large amounts of experience to learn an individual task.
While in principle meta-reinforcement learning (meta-RL) algorithms enable agents to learn new skills from small amounts of experience, several major challenges preclude their practicality.
Current methods rely heavily on on-policy experience, limiting their sample efficiency.
They also lack mechanisms to reason about task uncertainty when adapting to new tasks, limiting their effectiveness in sparse reward problems.
In this paper, we address these challenges by developing an off-policy meta-RL algorithm that disentangles task inference and control.
In our approach, we perform online probabilistic filtering of latent task variables to infer how to solve a new task from small amounts of experience.
This probabilistic interpretation enables posterior sampling for structured and efficient exploration.
We demonstrate how to integrate these task variables with off-policy RL algorithms to achieve both meta-training and adaptation efficiency.
Our method outperforms prior algorithms in sample efficiency by 20-100X as well as in asymptotic performance on several meta-RL benchmarks.

assistant: text: This paper proposes using off-policy RL during the meta-training time to greatly improve sample efficiency of Meta-RL methods.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Beyond Classical Diffusion: Ballistic Graph Neural Network

Paper content:
This paper presents the ballistic graph neural network.
Ballistic graph neural network tackles the weight distribution from a transportation perspective and has many different properties comparing to the traditional graph neural network pipeline.
The ballistic graph neural network does not require to calculate any eigenvalue.
The filters propagate exponentially faster($\sigma^2 \sim T^2$) comparing to traditional graph neural network($\sigma^2 \sim T$).
We use a perturbed coin operator to perturb and optimize the diffusion rate.
Our results show that by selecting the diffusion speed, the network can reach a similar accuracy with fewer parameters.
We also show the perturbed filters act as better representations comparing to pure ballistic ones.
We provide a new perspective of training graph neural network, by adjusting the diffusion rate, the neural network's performance can be improved.

assistant: text: The paper proposes to cope with the speed of diffusion problem by introducing ballistic walk.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Why do These Match? Explaining the Behavior of Image Similarity Models

Paper content:
Explaining a deep learning model can help users understand its behavior and allow researchers to discern its shortcomings.
Recent work has primarily focused on explaining models for tasks like image classification or visual question answering.
In this paper, we introduce an explanation approach for image similarity models, where a model's output is a score measuring the similarity of two inputs rather than a classification.
In this task, an explanation depends on both of the input images, so standard methods do not apply.
We propose an explanation method that pairs a saliency map identifying important image regions with an attribute that best explains the match.
We find that our explanations provide additional information not typically captured by saliency maps alone, and can also improve performance on the classic task of attribute recognition.
Our approach's ability to generalize is demonstrated on two datasets from diverse domains, Polyvore Outfits and Animals with Attributes 2.

assistant: text: Introduces method for image similarity model explanation which identifies attributes that contribute positively to the similarity score and pairs them with a generated saliency map.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Scalable Neural Learning for Verifiable Consistency with Temporal Specifications

Paper content:
Formal verification of machine learning models has attracted attention recently, and significant progress has been made on proving simple properties like robustness to small perturbations of the input features.
In this context, it has also been observed that folding the verification procedure into training makes it easier to train verifiably robust models.
In this paper, we extend the applicability of verified training by extending it to (1) recurrent neural network architectures and (2) complex specifications that go beyond simple adversarial robustness, particularly specifications that capture temporal properties like requiring that a robot periodically visits a charging station or that a language model always produces sentences of bounded length.
Experiments show that while models trained using standard training often violate desired specifications, our verified training method produces models that both perform well (in terms of test error or reward) and can be shown to be provably consistent with specifications.

assistant: text: This paper extends interval bound propagation to recurrent computation and auto-regressive models, introduces and extends Signal Temporal Logic for specifying temporal contraints, and provides proof that STL with bound propagation can ensure neural models conform to temporal specification.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Frequency-based Search-control in Dyna

Paper content:
Model-based reinforcement learning has been empirically demonstrated as a successful strategy to improve sample efficiency.
Particularly, Dyna architecture, as an elegant model-based architecture integrating learning and planning, provides huge flexibility of using a model.
One of the most important components in Dyna is called search-control, which refers to the process of generating state or state-action pairs from which we query the model to acquire simulated experiences.
Search-control is critical to improve learning efficiency.
In this work, we propose a simple and novel search-control strategy by searching high frequency region on value function.
Our main intuition is built on Shannon sampling theorem from signal processing, which indicates that a high frequency signal requires more samples to reconstruct.
We empirically show that a high frequency function is more difficult to approximate.
This suggests a search-control strategy: we should use states in high frequency region of the value function to query the model to acquire more samples.
We develop a simple strategy to locally measure the frequency of a function by gradient norm, and provide theoretical justification for this approach.
We then apply our strategy to search-control in Dyna, and conduct experiments to show its property and effectiveness on benchmark domains.

assistant: text: The authors propose to do sampling in the high-frequency domain to increase the sample efficiency
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Bayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning with Clairvoyant Experts

Paper content:
Informed and robust decision making in the face of uncertainty is critical for robots that perform physical tasks alongside people.
We formulate this as a Bayesian Reinforcement Learning problem over latent Markov Decision Processes (MDPs).
While Bayes-optimality is theoretically the gold standard, existing algorithms do not scale well to continuous state and action spaces.
We propose a scalable solution that builds on the following insight: in the absence of uncertainty, each latent MDP is easier to solve.
We split the challenge into two simpler components.
First, we obtain an ensemble of clairvoyant experts and fuse their advice to compute a baseline policy.
Second, we train a Bayesian residual policy to improve upon the ensemble's recommendation and learn to reduce uncertainty.
Our algorithm, Bayesian Residual Policy Optimization (BRPO), imports the scalability of policy gradient methods as well as the initialization from prior models.
BRPO significantly improves the ensemble of experts and drastically outperforms existing adaptive RL methods.

assistant: text: In this paper, the authors motivate and propose a learning algorithm, called Bayesian Residual Policy Optimization (BRPO), for Bayesian reinforcement learning problems.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: RL-ST: Reinforcing Style, Fluency and Content Preservation for Unsupervised Text Style Transfer

Paper content:
Unsupervised text style transfer is the task of re-writing text of a given style into a target style without using a parallel corpus of source style and target style sentences for training.
Style transfer systems are evaluated on their ability to generate sentences that
1) possess the target style,
2) are fluent and natural sounding, and
3) preserve the non-stylistic parts (content) of the source sentence.
We train a reinforcement learning (RL) based unsupervised style transfer system that incorporates rewards for the above measures, and describe novel rewards shaping methods for the same.
Our approach does not attempt to disentangle style and content, and leverages the power of massively pre-trained language models as well as the Transformer.
Our system significantly outperforms existing state-of-art systems based on human as well as automatic evaluations on target style, fluency and content preservation as well as on overall success of style transfer, on a variety of datasets.

assistant: text: Introduces an RL-based method which leverages a pre-trained language model to transfer text style, without a disentanglement objective, while using style-transfer generations from another model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Zero-Shot Medical Image Artifact Reduction

Paper content:
Medical images may contain various types of artifacts with different patterns and mixtures, which depend on many factors such as scan setting, machine condition, patients’ characteristics, surrounding environment, etc.
However, existing deep learning based artifact reduction methods are restricted by their training set with specific predetermined artifact type and pattern.
As such, they have limited clinical adoption.
In this paper, we introduce a “Zero-Shot” medical image Artifact Reduction (ZSAR) framework, which leverages the power of deep learning but without using general pre-trained networks or any clean image reference.
Specifically, we utilize the low internal visual entropy of an image and train a light-weight image-specific artifact reduction network to reduce artifacts in an image at test-time.
We use Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) as vehicles to show that ZSAR can reduce artifacts better than state-of-the-art both qualitatively and quantitatively, while using shorter execution time.
To the best of our knowledge, this is the first deep learning framework that reduces artifacts in medical images without using a priori training set.

assistant: text: We introduce a “Zero-Shot” medical image Artifact Reduction framework, which leverages the power of deep learning but without using general pre-trained networks or any clean image reference.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: In-Domain Representation Learning For Remote Sensing

Paper content:
Given the importance of remote sensing, surprisingly little attention has been paid to it by the representation learning community.
To address it and to speed up innovation in this domain, we provide simplified access to 5 diverse remote sensing datasets in a standardized form.
We specifically explore in-domain representation learning and address the question of "what characteristics should a dataset have to be a good source for remote sensing representation learning".
The established baselines achieve state-of-the-art performance on these datasets.

assistant: text: Exploration of in-domain representation learning for remote sensing datasets.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step

Paper content:
Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion.
Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost.
GANs are designed to reach a Nash equilibrium at which each player cannot reduce their cost without changing the other players’ parameters.
One useful approach for the theory of GANs is to show that a divergence between the training distribution and the model distribution obtains its minimum value at equilibrium.
Several recent research directions have been motivated by the idea that this divergence is the primary guide for the learning process and that every step of learning should decrease the divergence.
We show that this view is overly restrictive.
During GAN training, the discriminator provides learning signal in situations where the gradients of the divergences between distributions would not be useful.
We provide empirical counterexamples to the view of GAN training as divergence minimization.
Specifically, we demonstrate that GANs are able to learn distributions in situations where the divergence minimization point of view predicts they would fail.
We also show that gradient penalties motivated from the divergence minimization perspective are equally helpful when applied in other contexts in which the divergence minimization perspective does not predict they would be helpful.
This contributes to a growing body of evidence that GAN training may be more usefully viewed as approaching Nash equilibria via trajectories that do not necessarily minimize a specific divergence at each step.

assistant: text: We find evidence that divergence minimization may not be an accurate characterization of GAN training.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Hierarchical Complement Objective Training

Paper content:
Hierarchical label structures widely exist in many machine learning tasks, ranging from those with explicit label hierarchies such as image classification to the ones that have latent label hierarchies such as semantic segmentation.
Unfortunately, state-of-the-art methods often utilize cross-entropy loss which in-explicitly assumes the independence among class labels.
Motivated by the fact that class members from the same hierarchy need to be similar to each others, we design a new training diagram called Hierarchical Complement Objective Training (HCOT).
In HCOT, in addition to maximizing the probability of the ground truth class, we also neutralize the probabilities of rest of the classes in a hierarchical fashion, making the model take advantage of the label hierarchy explicitly.
We conduct our method on both image classification and semantic segmentation.
Results show that HCOT outperforms state-of-the-art models in CIFAR100, Imagenet, and PASCAL-context.
Our experiments also demonstrate that HCOT can be applied on tasks with latent label hierarchies, which is a common characteristic in many machine learning tasks.

assistant: text: We propose Hierarchical Complement Objective Training, a novel training paradigm to effectively leverage category hierarchy in the labeling space on both image classification and semantic segmentation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generative Question Answering: Learning to Answer the Whole Question

Paper content:
Discriminative  question  answering  models  can  overfit  to  superficial  biases  in datasets,  because their loss function saturates when any clue makes the answer likely.
We introduce generative models of the joint distribution of questions and answers, which are trained to explain the whole question, not just to answer it.
Our  question  answering  (QA)  model  is  implemented  by  learning  a  prior  over answers,  and  a  conditional  language  model  to  generate  the  question  given  the answer—allowing scalable and interpretable many-hop reasoning as the question is generated word-by-word.
Our model achieves competitive performance with specialised discriminative models on the SQUAD and CLEVR benchmarks, indicating that it is a more general architecture for language understanding and reasoning than previous work.
The model greatly improves generalisation both from biased training data and to adversarial testing data, achieving a new state-of-the-art on ADVERSARIAL SQUAD.
We will release our code.

assistant: text: Question answering models that model the joint distribution of questions and answers can learn more than discriminative models
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Accelerating Convolutional Neural Networks using Iterative Two-Pass Decomposition

Paper content:
We present the iterative two-pass decomposition flow to accelerate existing convolutional neural networks (CNNs).
The proposed rank selection algorithm can effectively determine the proper ranks of the target convolutional layers for the low rank approximation.
Our two-pass CP-decomposition helps prevent from the instability problem.
The iterative flow makes the decomposition of the deeper networks systematic.
The experiment results shows that VGG16 can be accelerated with a 6.2x measured speedup while the accuracy drop remains only 1.2%.
Deep learning has become of vital importance in a variety of artificial intelligence applications.
Recently, convolutional neural networks (CNNs) have been widely applied to have the breakthrough in improving the recognition accuracy for challenging computer vision tasks such as image classification, localization, object detection, and so on BID19 ; BID11 ; BID20 ; BID3 ).
However, those significant achievements using CNNs come with the cost of larger network size and higher computational complexity, which leads to an increasing difficulty for deploying to resource constrained edge devices, or even for the fast computation on the cloud servers.
This paper addresses the acceleration of the existing CNN models to cope with such burden.
The iterative two-pass decomposition flow has been presented to accelerate existing deep CNNs.
Our two-pass decomposition effectively prevents from the CP instability.
The Rank Selection algorithm provides the fine-grained rank configuration to achieve the target speedup while maintaining the accuracy.
The experiment results show that VGG16 can be accelerated by 6.2 times with the accuracy drop of only 1.20% and the size reduction of 85%.
In addition, ResNet50 can be speeded up by 1.35 times with the accuracy drop of 1.51% and the size reduction of 48%.The
future works include the improvement of the grouping scheme and the decomposing order, and a smarter rank selection with non-linear fitness estimation. In
addition, accelerating 1×1 convolutional layers will also be considered for the further improvement on the advanced CNNs.APPENDIX A FIG6 shows the initial fitnesses of the baseline rank configuration and fitness-based configuration. Note
that the layer order is sorted by the fitnesses of the baseline rank configuration. Because
there is only a slight difference between the sorting of the two configuration, the fitness-based grouping scheme is based on the sorting of the baseline rank configuration. TAB1

assistant: text: The paper proposes a novel workflow for acceleration and compression of CNNs and also proposes a way to determine the target rank of each layer given the target overall acceleration.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Fully differentiable full-atom protein backbone generation

Paper content:
The fast generation and refinement of protein backbones would constitute a major advancement to current methodology for the design and development of de novo proteins.
In this study, we train Generative Adversarial Networks (GANs) to generate fixed-length full-atom protein backbones, with the goal of sampling from the distribution of realistic 3-D backbone fragments.
We represent protein structures by pairwise distances between all backbone atoms, and present a method for directly recovering and refining the corresponding backbone coordinates in a differentiable manner.
We show that interpolations in the latent space of the generator correspond to smooth deformations of the output backbones, and that test set structures not seen by the generator during training exist in its image.
Finally, we perform sequence design, relaxation, and ab initio folding of a subset of generated structures, and show that in some cases we can recover the generated folds after forward-folding.
Together, these results suggest a mechanism for fast protein structure refinement and folding using external energy functions.

assistant: text: This paper presents an end-to-end approach for generating protein backbones using generative adversarial networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Image-guided Neural Object Rendering

Paper content:
We propose a learned image-guided rendering technique that combines the benefits of image-based rendering and GAN-based image synthesis.
The goal of our method is to generate photo-realistic re-renderings of reconstructed objects for virtual and augmented reality applications (e.g., virtual showrooms, virtual tours and sightseeing, the digital inspection of historical artifacts).
A core component of our work is the handling of view-dependent effects.
Specifically, we directly train an object-specific deep neural network to synthesize the view-dependent appearance of an object.
As input data we are using an RGB video of the object.
This video is used to reconstruct a proxy geometry of the object via multi-view stereo.
Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering.
This warping assumes diffuse surfaces, in case of view-dependent effects, such as specular highlights, it leads to artifacts.
To this end, we propose EffectsNet, a deep neural network that predicts view-dependent effects.
Based on these estimations, we are able to convert observed images to diffuse images.
These diffuse images can be projected into other views.
In the target view, our pipeline reinserts the new view-dependent effects.
To composite multiple reprojected images to a final output, we learn a composition network that outputs photo-realistic results.
Using this image-guided approach, the network does not have to allocate capacity on ``remembering'' object appearance, instead it learns how to combine the appearance of captured images.
We demonstrate the effectiveness of our approach both qualitatively and quantitatively on synthetic as well as on real data.

assistant: text: This submission proposes a method to handle view-dependent effects in neural rendering, which improves the robustness of existing neural rendering methods.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Mitigating Posterior Collapse in Strongly Conditioned Variational Autoencoders

Paper content:
Training conditional generative latent-variable models is challenging in scenarios where the conditioning signal is very strong and the decoder is expressive enough to generate a plausible output given only the condition; the generative model tends to ignore the latent variable, suffering from posterior collapse.
We find, and empirically show, that one of the major reasons behind posterior collapse is rooted in the way that generative models are conditioned, i.e., through concatenation of the latent variable and the condition
. To mitigate this problem, we propose to explicitly make the latent variables depend on the condition by unifying the conditioning and latent variable sampling, thus coupling them so as to prevent the model from discarding the root of variations
. To achieve this, we develop a conditional Variational Autoencoder architecture that learns a distribution not only of the latent variables, but also of the condition, the latter acting as prior on the former
. Our experiments on the challenging tasks of conditional human motion prediction and image captioning demonstrate the effectiveness of our approach at avoiding posterior collapse
. Video results of our approach are anonymously provided in http://bit.ly/iclr2020

assistant: text: This paper argues that when the decoder is conditioned on the concatenation of latent variables and auxiliary information, then posterior collapse is more likely than in vanilla VAE.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Overcoming Catastrophic Forgetting via Hessian-free Curvature Estimates

Paper content:
Learning neural networks with gradient descent over a long sequence of tasks is problematic as their fine-tuning to new tasks overwrites the network weights that are important for previous tasks.
This leads to a poor performance on old tasks – a phenomenon framed as catastrophic forgetting.
While early approaches use task rehearsal and growing networks that both limit the scalability of the task sequence orthogonal approaches build on regularization.
Based on the Fisher information matrix (FIM) changes to parameters that are relevant to old tasks are penalized, which forces the task to be mapped into the available remaining capacity of the network.
This requires to calculate the Hessian around a mode, which makes learning tractable.
In this paper, we introduce Hessian-free curvature estimates as an alternative method to actually calculating the Hessian.
In contrast to previous work, we exploit the fact that most regions in the loss surface are flat and hence only calculate a Hessian-vector-product around the surface that is relevant for the current task.
Our experiments show that on a variety of well-known task sequences we either significantly outperform or are en par with previous work.

assistant: text: The paper proposes an approximate Laplace's method in neural network training in the continual learning setting with a low space complexity.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Refining the variational posterior through iterative optimization

Paper content:
Variational inference (VI) is a popular approach for approximate Bayesian inference that is particularly promising for highly parameterized models such as deep neural networks.
A key challenge of variational inference is to approximate the posterior over model parameters with a distribution that is simpler and tractable yet sufficiently expressive.
In this work, we propose a method for training highly flexible variational distributions by starting with a coarse approximation and iteratively refining it.
Each refinement step makes cheap, local adjustments and only requires optimization of simple variational families.
We demonstrate theoretically that our method always improves a bound on the approximation (the Evidence Lower BOund) and observe this empirically across a variety of benchmark tasks.
In experiments, our method consistently outperforms recent variational inference methods for deep learning in terms of log-likelihood and the ELBO.
We see that the gains are further amplified on larger scale models, significantly outperforming standard VI and deep ensembles on residual networks on CIFAR10.

assistant: text: The paper proposes an algorithm to increase the flexibility of the variational posterior in Bayesian neural networks through iterative optimization.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Do Deep Neural Networks for Segmentation Understand Insideness?

Paper content:
Image segmentation aims at grouping pixels that belong to the same object or region.
At the heart of image segmentation lies the problem of determining whether a pixel is inside or outside a region, which we denote as the "insideness" problem.
Many Deep Neural Networks (DNNs) variants excel in segmentation benchmarks, but regarding insideness, they have not been well visualized or understood: What representations do DNNs use to address the long-range relationships of insideness?
How do architectural choices affect the learning of these representations?
In this paper, we take the reductionist approach by analyzing DNNs solving the insideness problem in isolation, i.e. determining the inside of closed (Jordan) curves.
We demonstrate analytically that state-of-the-art feed-forward and recurrent architectures can implement solutions of the insideness problem for any given curve.
Yet, only recurrent networks could  learn these general solutions when the training enforced a specific "routine" capable of breaking down the long-range relationships.
Our results highlights the need for new training strategies that decompose the learning into appropriate stages, and that lead to the general class of solutions necessary for DNNs to understand insideness.

assistant: text: This paper introduces insideness to study semantic segmentation in deep learning era, and the results can help models generalize better.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: CONFEDERATED MACHINE LEARNING ON HORIZONTALLY AND VERTICALLY SEPARATED MEDICAL DATA FOR LARGE-SCALE HEALTH SYSTEM INTELLIGENCE

Paper content:
A patient’s health information is generally fragmented across silos.
Though it is technically feasible to unite data for analysis in a manner that underpins a rapid learning healthcare system, privacy concerns and regulatory barriers limit data centralization.
Machine learning can be conducted in a federated manner on patient datasets with the same set of variables, but separated across sites of care.
But federated learning cannot handle the situation where different data types for a given
patient are separated vertically across different organizations.
We call methods that enable machine learning model training on data separated by two or more degrees “confederated machine learning.”
We built and evaluated a confederated machine
learning
model to stratify the risk of accidental falls among the elderly.

assistant: text: A "confederated" machine learning method that learns across divides in medical data separated both horizontally and vertically.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints

Paper content:
In most practical settings and theoretical analyses, one assumes that a model can be trained until convergence.
However, the growing complexity of machine learning datasets and models may violate such assumptions.
Indeed, current approaches for hyper-parameter tuning and neural architecture search tend to be limited by practical resource constraints.
Therefore, we introduce a formal setting for studying training under the non-asymptotic, resource-constrained regime, i.e., budgeted training.
We analyze the following problem: "given a dataset, algorithm, and fixed resource budget, what is the best achievable performance?"
We focus on the number of optimization iterations as the representative resource.
Under such a setting, we show that it is critical to adjust the learning rate schedule according to the given budget.
Among budget-aware learning schedules, we find simple linear decay to be both robust and high-performing.
We support our claim through extensive experiments with state-of-the-art models on ImageNet (image classification), Kinetics (video classification), MS COCO (object detection and instance segmentation), and Cityscapes (semantic segmentation).
We also analyze our results and find that the key to a good schedule is budgeted convergence, a phenomenon whereby the gradient vanishes at the end of each allowed budget.
We also revisit existing approaches for fast convergence and show that budget-aware learning schedules readily outperform such approaches under (the practical but under-explored) budgeted training setting.

assistant: text: This work presents a technique for tuning the learning rate for Neural Network training when under a fixed number of epochs.