ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Noisy $\ell^{0}$-Sparse Subspace Clustering on Dimensionality Reduced Data

Paper content:
High-dimensional data often lie in or close to low-dimensional subspaces.
Sparse subspace clustering methods with sparsity induced by L0-norm, such as L0-Sparse Subspace Clustering (L0-SSC), are demonstrated to be more effective than its L1 counterpart such as Sparse Subspace Clustering (SSC).
However, these L0-norm based subspace clustering methods are restricted to clean data that lie exactly in subspaces.
Real data often suffer from noise and they may lie close to subspaces.
We propose noisy L0-SSC to handle noisy data so as to improve the robustness.
We show that the optimal solution to the optimization problem of noisy L0-SSC achieves subspace detection property (SDP), a key element with which data from different subspaces are separated, under deterministic and randomized models.
Our results provide theoretical guarantee on the correctness of noisy L0-SSC in terms of SDP on noisy data.
We further propose Noisy-DR-L0-SSC which provably recovers the subspaces on dimensionality reduced data.
Noisy-DR-L0-SSC first projects the data onto a lower dimensional space by linear transformation, then performs noisy L0-SSC on the dimensionality reduced data so as to improve the efficiency.
The experimental results demonstrate the effectiveness of noisy L0-SSC and Noisy-DR-L0-SSC.

assistant: text: We propose Noisy-DR-L0-SSC (Noisy Dimension Reduction L0-Sparse Subspace Clustering) to efficiently partition noisy data in accordance to their underlying subspace structure.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Harmonic Unpaired Image-to-image Translation

Paper content:
The recent direction of unpaired image-to-image translation is on one hand very exciting as it alleviates the big burden in obtaining label-intensive pixel-to-pixel supervision, but it is on the other hand not fully satisfactory due to the presence of artifacts and degenerated transformations.
In this paper, we take a manifold view of the problem by introducing a smoothness term over the sample graph to attain harmonic functions to enforce consistent mappings during the translation.
We develop HarmonicGAN to learn bi-directional translations between the source and the target domains.
With the help of similarity-consistency, the inherent self-consistency property of samples can be maintained.
Distance metrics defined on two types of features including histogram and CNN are exploited.
Under an identical problem setting as CycleGAN, without additional manual inputs and only at a small training-time cost, HarmonicGAN demonstrates a significant qualitative and quantitative improvement over the state of the art, as well as improved interpretability.
We show experimental results in a number of applications including medical imaging, object transfiguration, and semantic labeling.
We outperform the competing methods in all tasks, and for a medical imaging task in particular our method turns CycleGAN from a failure to a success, halving the mean-squared error, and generating images that radiologists prefer over competing methods in 95% of cases.

assistant: text: Smooth regularization over sample graph for unpaired image-to-image translation results in significantly improved consistency
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning to Compute Word Embeddings On the Fly

Paper content:
Words in natural language follow a Zipfian distribution whereby some words are frequent but most are rare.
Learning representations for words in the ``long tail'' of this distribution requires enormous amounts of data.
Representations of rare words trained directly on end tasks are usually poor, requiring us to pre-train embeddings on external data, or treat all rare words as out-of-vocabulary words with a unique representation.
We provide a method for predicting embeddings of rare words on the fly from small amounts of auxiliary data with a network trained end-to-end for the downstream task.
We show that this improves results against baselines where embeddings are trained on the end task for reading comprehension, recognizing textual entailment and language modeling.

assistant: text: We propose a method to deal with rare words by computing their embedding from definitions.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Models in the Wild: On Corruption Robustness of NLP Systems

Paper content:
Natural Language Processing models lack a unified approach to robustness testing.
In this paper we introduce WildNLP - a framework for testing model stability in a natural setting where text corruptions such as keyboard errors or misspelling occur.
We compare robustness of models from 4 popular NLP tasks: Q&A, NLI, NER and Sentiment Analysis by testing their performance on aspects introduced in the framework.
In particular, we focus on a comparison between recent state-of-the- art text representations and non-contextualized word embeddings.
In order to improve robust- ness, we perform adversarial training on se- lected aspects and check its transferability to the improvement of models with various cor- ruption types.
We find that the high perfor- mance of models does not ensure sufficient robustness, although modern embedding tech- niques help to improve it.
We release cor- rupted datasets and code for WildNLP frame- work for the community.

assistant: text: We compare robustness of models from 4 popular NLP tasks: Q&A, NLI, NER and Sentiment Analysis by testing their performance on perturbed inputs.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Diversity is All You Need: Learning Skills without a Reward Function

Paper content:
Intelligent creatures can explore their environments and learn useful skills without supervision.
In this paper, we propose ``Diversity is All You Need''(DIAYN), a method for learning useful skills without a reward function.
Our proposed method learns skills by maximizing an information theoretic objective using a maximum entropy policy.
On a variety of simulated robotic tasks, we show that this simple objective results in the unsupervised emergence of diverse skills, such as walking and jumping.
In a number of reinforcement learning benchmark environments, our method is able to learn a skill that solves the benchmark task despite never receiving the true task reward.
We show how pretrained skills can provide a good parameter initialization for downstream tasks, and can be composed hierarchically to solve complex, sparse reward tasks.
Our results suggest that unsupervised discovery of skills can serve as an effective pretraining mechanism for overcoming challenges of exploration and data efficiency in reinforcement learning.

assistant: text: We propose an algorithm for learning useful skills without a reward function, and show how these skills can be used to solve downstream tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Achieving morphological agreement with Concorde

Paper content:
Neural conversational models are widely used in applications like personal assistants and chat bots.
These models seem to give better performance when operating on word level.
However, for fusion languages like French, Russian and Polish vocabulary size sometimes become infeasible since most of the words have lots of word forms.
We propose a neural network architecture for transforming normalized text into a grammatically correct one.
Our model efficiently employs correspondence between normalized and target words and significantly outperforms character-level models while being 2x faster in training and 20\% faster at evaluation.
We also propose a new pipeline for building conversational models: first generate a normalized answer and then transform it into a grammatically correct one using our network.
The proposed pipeline gives better performance than character-level conversational models according to assessor testing.

assistant: text: Proposed architecture to solve morphological agreement task
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generative Restricted Kernel Machines

Paper content:
We introduce a novel framework for generative models based on Restricted Kernel Machines (RKMs) with multi-view generation and uncorrelated feature learning capabilities, called Gen-RKM.
To incorporate multi-view generation, this mechanism uses a shared representation of data from various views.
The mechanism is flexible to incorporate both kernel-based, (deep) neural network and convolutional based models within the same setting.
To update the parameters of the network, we propose a novel training procedure which jointly learns the features and shared representation.
Experiments demonstrate the potential of the framework through qualitative evaluation of generated samples.

assistant: text: Gen-RKM: a novel framework for generative models using Restricted Kernel Machines with multi-view generation and uncorrelated feature learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Convolutional Tensor-Train LSTM for Long-Term Video Prediction

Paper content:
Long-term video prediction is highly challenging since it entails simultaneously capturing spatial and temporal information across a long range of image frames.Standard recurrent models are ineffective since they are prone to error propagation and cannot effectively capture higher-order correlations.
A potential solution is to extend to higher-order spatio-temporal recurrent models.
However, such a model requires  a  large number of parameters and operations, making it intractable  to learn in practice and is prone to overfitting.
In this work, we propose convolutional tensor-train LSTM (Conv-TT-LSTM), which  learns higher-orderConvolutional LSTM (ConvLSTM) efficiently using convolutional  tensor-train decomposition (CTTD).
Our proposed model naturally incorporates higher-order spatio-temporal information at a small cost of memory and computation by using efficient low-rank tensor representations.
We evaluate our model on Moving-MNIST and KTH datasets and show improvements over standard ConvLSTM and better/comparable results to other ConvLSTM-based approaches, but with much fewer parameters.

assistant: text: we propose convolutional tensor-train LSTM,  which learns higher-order Convolutional LSTM efficiently using convolutional tensor-train decomposition.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Interactive Image Generation Using Scene Graphs

Paper content:
Recent years have witnessed some exciting developments in the domain of generating images from scene-based text descriptions.
These approaches have primarily focused on generating images from a static text description and are limited to generating images in a single pass.
They are unable to generate an image interactively based on an incrementally additive text description (something that is more intuitive and similar to the way we describe an image).
We propose a method to generate an image incrementally based on a sequence of graphs of scene descriptions (scene-graphs).
We propose a recurrent network architecture that preserves the image content generated in previous steps and modifies the cumulative image as per the newly provided scene information.
Our model utilizes Graph Convolutional Networks (GCN) to cater to variable-sized scene graphs along with Generative Adversarial image translation networks to generate realistic multi-object images without needing any intermediate supervision during training.
We experiment with Coco-Stuff dataset which has multi-object images along with annotations describing the visual scene and show that our model significantly outperforms other approaches on the same dataset in generating visually consistent images for incrementally growing scene graphs.

assistant: text: Interactively generating image from incrementally growing scene graphs in multiple steps using GANs while preserving the contents of image generated in previous steps
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Network Reparameterization for Unseen Class Categorization

Paper content:
Many problems with large-scale labeled training data have been impressively solved by deep learning.
However, Unseen Class Categorization (UCC) with minimal information provided about target classes is the most commonly encountered setting in industry, which remains a challenging research problem in machine learning.
Previous approaches to UCC either fail to generate a powerful discriminative feature extractor or fail to learn a flexible classifier that can be easily adapted to unseen classes.
In this paper, we propose to address these issues through network reparameterization, \textit{i.e.}, reparametrizing the learnable weights of a network as a function of other variables, by which we decouple the feature extraction part and the classification part of a deep classification model to suit the special setting of UCC, securing both strong discriminability and excellent adaptability.
Extensive experiments for UCC on several widely-used benchmark datasets in the settings of zero-shot and few-shot learning demonstrate that, our method with network reparameterization achieves state-of-the-art performance.

assistant: text: A unified frame for both few-shot learning and zero-shot learning based on network reparameterization
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adversarially Regularized Autoencoders

Paper content:
While autoencoders are a key technique in representation learning for continuous structures, such as images or wave forms, developing general-purpose autoencoders for discrete structures, such as text sequence or discretized images, has proven to be more challenging.
In particular, discrete inputs make it more difficult to learn a smooth encoder that preserves the complex local relationships in the input space.
In this work, we propose an adversarially regularized autoencoder (ARAE) with the goal of learning more robust discrete-space representations.
ARAE jointly trains both a rich discrete-space encoder, such as an RNN, and a simpler continuous space generator function, while using generative adversarial network (GAN) training to constrain the distributions to be similar.
This method yields a smoother contracted code space that maps similar inputs to nearby codes, and also an implicit latent variable GAN model for generation.
Experiments on text and discretized images demonstrate that the GAN model produces clean interpolations and captures the multimodality of the original space, and that the autoencoder produces improvements in semi-supervised learning as well as state-of-the-art results in unaligned text style transfer task using only a shared continuous-space representation.

assistant: text: Adversarially Regularized Autoencoders learn smooth representations of discrete structures allowing for interesting results in text generation, such as unaligned style transfer, semi-supervised learning, and latent space interpolation and arithmetic.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generative Adversarial Networks for Extreme Learned Image Compression

Paper content:
We propose a framework for extreme learned image compression based on Generative Adversarial Networks (GANs), obtaining visually pleasing images at significantly lower bitrates than previous methods.
This is made possible through our GAN formulation of learned compression combined with a generator/decoder which operates on the full-resolution image and is trained in combination with a multi-scale discriminator.
Additionally, if a semantic label map of the original image is available, our method can fully synthesize unimportant regions in the decoded image such as streets and trees from the label map, therefore only requiring the storage of the preserved region and the semantic label map.
A user study confirms that for low bitrates, our approach is preferred to state-of-the-art methods, even when they use more than double the bits.

assistant: text: GAN-based extreme image compression method using less than half the bits of the SOTA engineered codec while preserving visual quality
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deepström Networks

Paper content:
Recent work has focused on combining kernel methods and deep learning.
With this in mind, we introduce Deepström networks -- a new architecture of neural networks which we use to replace top dense layers of standard convolutional architectures with an approximation of a kernel function by relying on the Nyström approximation.
Our approach is easy highly flexible.
It is compatible with any kernel function and it allows exploiting multiple kernels.
We show that Deepström networks reach state-of-the-art performance on standard datasets like SVHN and CIFAR100.
One benefit of the method lies in its limited number of learnable parameters which make it particularly suited for small training set sizes, e.g. from 5 to 20 samples per class.
Finally we illustrate two ways of using multiple kernels, including a multiple Deepström  setting, that exploits a kernel on each feature map output by the convolutional part of the model.

assistant: text: A new neural architecture where top dense layers of standard convolutional architectures are replaced with an approximation of a kernel function by relying on the Nyström approximation.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generative Models for Low-Dimensional Video Representation and Compressive Sensing

Paper content:
Generative priors have become highly effective in solving inverse problems including denoising, inpainting, and reconstruction from few and noisy measurements.
With a generative model we can represent an image with a much lower dimensional latent codes.
In the context of compressive sensing, if the unknown image belongs to the range of a pretrained generative network, then we can recover the image by estimating the underlying compact latent code from the available measurements.
However, recent studies revealed that even untrained deep neural networks can work as a prior for recovering natural images.
These approaches update the network weights keeping latent codes fixed to reconstruct the target image from the given measurements.
In this paper, we optimize over network weights and latent codes to use untrained generative network as prior for video compressive sensing problem.
We show that by optimizing over latent code, we can additionally get concise representation of the frames which retain the structural similarity of the video frames.
We also apply low-rank constraint on the latent codes to represent the video sequences in even lower dimensional latent space.
We empirically show that our proposed methods provide better or comparable accuracy and low computational complexity compared to the existing methods.

assistant: text: Recover videos from compressive measurements by learning a low-dimensional (low-rank) representation directly from measurements while training a deep generator.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Continuous Graph Flow

Paper content:
In this paper, we propose Continuous Graph Flow, a generative continuous flow based method that aims to model complex distributions of graph-structured data.
Once learned, the model can be applied to an arbitrary graph, defining a probability density over the random variables represented by the graph.
It is formulated as an ordinary differential equation system with shared and reusable functions that operate over the graphs.
This leads to a new type of neural graph message passing scheme that performs continuous message passing over time.
This class of models offers several advantages: a flexible representation that can generalize to variable data dimensions; ability to model dependencies in complex data distributions; reversible and memory-efficient; and exact and efficient computation of the likelihood of the data.
We demonstrate the effectiveness of our model on a diverse set of generation tasks across different domains: graph generation, image puzzle generation, and layout generation from scene graphs.
Our proposed model achieves significantly better performance compared to  state-of-the-art models.

assistant: text: Graph generative models based on generalization of message passing to continuous time using ordinary differential equations
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: PROTOTYPE-ASSISTED ADVERSARIAL LEARNING FOR UNSUPERVISED DOMAIN ADAPTATION

Paper content:
This paper presents a generic framework to tackle the crucial class mismatch problem in unsupervised domain adaptation (UDA) for multi-class distributions.
Previous adversarial learning methods condition domain alignment only on pseudo labels, but noisy and inaccurate pseudo labels may perturb the multi-class distribution embedded in probabilistic predictions, hence bringing insufficient alleviation to the latent mismatch problem.
Compared with pseudo labels, class prototypes are more accurate and reliable since they summarize over all the instances and are  able  to  represent  the  inherent  semantic  distribution  shared  across  domains.
Therefore, we propose a novel Prototype-Assisted Adversarial Learning (PAAL) scheme, which incorporates instance probabilistic predictions and class prototypes together  to  provide  reliable  indicators  for  adversarial  domain  alignment.
With the PAAL scheme,  we align both the instance feature representations and class prototype  representations  to  alleviate  the  mismatch  among  semantically  different classes.
Also,  we exploit the class prototypes as proxy to minimize the within-class variance in the target domain to mitigate the mismatch among semantically similar classes.
With these novelties, we constitute a Prototype-Assisted Conditional Domain Adaptation (PACDA) framework which well tackles the class mismatch problem.
We demonstrate the good performance and generalization ability of the PAAL scheme and also PACDA framework on two UDA tasks, i.e., object recognition (Office-Home,ImageCLEF-DA, andOffice) and synthetic-to-real semantic segmentation (GTA5→CityscapesandSynthia→Cityscapes).

assistant: text: We propose a reliable conditional adversarial learning scheme along with a simple, generic yet effective framework for UDA tasks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: INFORMATION MAXIMIZATION AUTO-ENCODING

Paper content:
We propose the Information Maximization Autoencoder (IMAE), an information theoretic approach to simultaneously learn continuous and discrete representations in an unsupervised setting.
Unlike the Variational Autoencoder framework, IMAE starts from a stochastic encoder that seeks to map each input data to a hybrid discrete and continuous representation with the objective of maximizing the mutual information between the data and their representations.
A decoder is included to approximate the posterior distribution of the data given their representations, where a high fidelity approximation can be achieved by leveraging the informative representations.
We show that the proposed objective is theoretically valid and provides a principled framework for understanding the tradeoffs regarding informativeness of each representation factor, disentanglement of representations, and decoding quality.

assistant: text: Information theoretical approach for unsupervised learning of unsupervised learning of a hybrid of discrete and continuous representations,
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Characterizing convolutional neural networks with one-pixel signature

Paper content:
We propose a new representation, one-pixel signature, that can be used to reveal the characteristics of the convolution neural networks (CNNs).
Here, each CNN classifier is associated with a signature that is created by generating, pixel-by-pixel, an adversarial value that is the result of the largest change to the class prediction.
The one-pixel signature is agnostic to the design choices of CNN architectures such as type, depth, activation function, and how they were trained.
It can be computed efficiently for a black-box classifier without accessing the network parameters.
Classic networks such as LetNet, VGG, AlexNet, and ResNet demonstrate different characteristics in their signature images.
For application, we focus on the classifier backdoor detection problem where a CNN classifier has been maliciously inserted with an unknown Trojan.
We show the effectiveness of the one-pixel signature in detecting backdoored CNN.
Our proposed one-pixel signature representation is general and it can be applied in problems where discriminative classifiers, particularly neural network based, are to be characterized.

assistant: text: Cnvolutional neural networks characterization for backdoored classifier detection and understanding.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Succinct Source Coding of Deep Neural Networks

Paper content:
Deep neural networks have shown incredible performance for inference tasks in a variety of domains.
Unfortunately, most current deep networks are enormous cloud-based structures that require significant storage space, which limits scaling of deep learning as a service (DLaaS) and use for on-device augmented intelligence.
This paper finds algorithms that directly use lossless compressed representations of deep feedforward networks (with synaptic weights drawn from discrete sets), to perform inference without full decompression.
The basic insight that allows less rate than naive approaches is the recognition that the bipartite graph layers of feedforward networks have a kind of permutation invariance to the labeling of nodes, in terms of inferential operation and that the inference operation depends locally on the edges directly connected to it.
We also provide experimental results of our approach on the MNIST dataset.

assistant: text: This paper finds algorithms that directly use lossless compressed representations of deep feedforward networks, to perform inference without full decompression.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Finding a human-like classifier

Paper content:
There were many attempts to explain the trade-off between accuracy and adversarial robustness.
However,  there  was  no  clear  understanding  of  the  behaviors  of  a  robust  classifier  which  has human-like robustness.
We  argue  (1)  why  we  need  to  consider  adversarial  robustness  against  varying  magnitudes  of perturbations not only focusing on a fixed perturbation threshold, (2) why we need to use different method to generate adversarially perturbed samples that can be used to train a robust classifier and measure the robustness of classifiers and (3) why we need to prioritize adversarial accuracies with different magnitudes.
We introduce Lexicographical Genuine Robustness (LGR) of classifiers that combines the above requirements.
We also suggest a candidate oracle classifier called "Optimal Lexicographically Genuinely  Robust  Classifier  (OLGRC)"  that  prioritizes  accuracy  on  meaningful  adversarially perturbed  examples  generated  by  smaller  magnitude  perturbations.
The  training  algorithm  for estimating OLGRC requires lexicographical optimization unlike existing adversarial training methods.
To apply lexicographical optimization to neural network, we utilize Gradient Episodic Memory (GEM) which was originally developed for continual learning by preventing catastrophic forgetting.

assistant: text: We try to design and train a classifier whose adversarial robustness is more resemblance to robustness of human.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Lifelong Learning with Output Kernels

Paper content:
Lifelong learning poses considerable challenges in terms of effectiveness (minimizing prediction errors for all tasks) and overall computational tractability for real-time performance.
This paper addresses continuous lifelong multitask learning by jointly re-estimating the inter-task relations (\textit{output} kernel) and the per-task model parameters at each round, assuming data arrives in a streaming fashion.
We propose a novel algorithm called  \textit{Online Output Kernel Learning Algorithm} (OOKLA) for lifelong learning setting.
To avoid the memory explosion, we propose a robust budget-limited versions of the proposed algorithm that efficiently utilize the relationship between the tasks to bound the total number of representative examples in the support set.
In addition, we propose a two-stage budgeted scheme for efficiently tackling the task-specific budget constraints in lifelong learning.
Our empirical results over three datasets indicate superior AUC performance for OOKLA and its budget-limited cousins over strong baselines.

assistant: text: a novel approach for online lifelong learning using output kernels.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Rethinking the Value of Network Pruning

Paper content:
Network pruning is widely used for reducing the heavy computational cost of deep models.
A typical pruning algorithm is a three-stage pipeline, i.e., training (a large model), pruning and fine-tuning.
In this work, we make a rather surprising observation: fine-tuning a pruned model only gives comparable or even worse performance than training that model with randomly initialized weights.
Our results have several implications:
1) training a large, over-parameterized model is not necessary to obtain an efficient final model,
2) learned "important" weights of the large model are not necessarily useful for the small pruned model,
3) the pruned architecture itself, rather than a set of inherited weights, is what leads to the efficiency benefit in the final model, which suggests that some pruning algorithms could be seen as performing network architecture search.

assistant: text: In network pruning, fine-tuning a pruned model only gives comparable or worse performance than training it from scratch. This advocate a rethinking of existing pruning algorithms.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Becoming Cat People: Animal-like Human Experiences with a Sensory Augmenting Whisker Wearable

Paper content:
Humans have a natural curiosity to imagine what it feels like to exist as someone or something else.
This curiosity becomes even stronger for the pets we care for.
Humans cannot truly know what it is like to be our pets, but we can deepen our understanding of what it is like to perceive and explore the world like them.
We investigate how wearables can offer people animal perspective-taking opportunities to experience the world through animal senses that differ from those biologically natural to us.
To assess the potential of wearables in animal perspective-taking, we developed a sensory-augmenting wearable that gives wearers cat-like whiskers.
We then created a maze exploration experience where blindfolded participants utilized the whiskers to navigate the maze.
We draw on animal behavioral research to evaluate how the whisker activity supported authentically cat-like experiences, and discuss the implications of this work for future learning experiences.

assistant: text: This paper explores using wearable sensory augmenting technology to facilitate first-hand perspective-taking of what it is like to have cat-like whiskers.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Discrete Flows: Invertible Generative Models of Discrete Data

Paper content:
While normalizing flows have led to significant advances in modeling high-dimensional continuous distributions, their applicability to discrete distributions remains unknown.
In this paper, we show that flows can in fact be extended to discrete events---and under a simple change-of-variables formula not requiring log-determinant-Jacobian computations.
Discrete flows have numerous applications.
We display proofs of concept under 2 flow architectures: discrete autoregressive flows enable bidirectionality, allowing for example tokens in text to depend on both left-to-right and right-to-left contexts in an exact language model; and discrete bipartite flows (i.e., with layer structure from RealNVP) enable parallel generation such as exact nonautoregressive text modeling.

assistant: text: We extend autoregressive flows and RealNVP to discrete data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Parametric Adversarial Divergences are Good Task Losses for Generative Modeling

Paper content:
Generative modeling of high dimensional data like images is a notoriously difficult and ill-defined problem.
In particular, how to evaluate a learned generative model is unclear.
In this paper, we argue that *adversarial learning*, pioneered with generative adversarial networks (GANs), provides an interesting framework to implicitly define more meaningful task losses for unsupervised tasks, such as for generating "visually realistic" images.
By relating GANs and structured prediction under the framework of statistical decision theory, we put into light links between recent advances in structured prediction theory and the choice of the divergence in GANs.
We argue that the insights about the notions of "hard" and "easy" to learn losses can be analogously extended to adversarial divergences.
We also discuss the attractive properties of parametric adversarial divergences for generative modeling, and perform experiments to show the importance of choosing a divergence that reflects the final task.

assistant: text: Parametric adversarial divergences implicitly define more meaningful task losses for generative modeling, we make parallels with structured prediction to study the properties of these divergences and their ability to encode the task of interest.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Support-guided Adversarial Imitation Learning

Paper content:
We propose Support-guided Adversarial Imitation Learning (SAIL), a generic imitation learning framework that unifies support estimation of the expert policy with the family of Adversarial Imitation Learning (AIL) algorithms.
SAIL addresses two important challenges of AIL, including the implicit reward bias and potential training instability.
We also show that SAIL is at least as efficient as standard AIL.
In an extensive evaluation, we demonstrate that the proposed method effectively handles the reward bias and achieves better performance and training stability than other baseline methods on a wide range of benchmark control tasks.

assistant: text: We unify support estimation with the family of Adversarial Imitation Learning algorithms into Support-guided Adversarial Imitation Learning, a more robust and stable imitation learning framework.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: On the Linguistic Capacity of Real-time Counter Automata

Paper content:
While counter machines have received little attention in theoretical computer science since the 1960s, they have recently achieved a newfound relevance to the field of natural language processing (NLP).
Recent work has suggested that some strong-performing recurrent neural networks utilize their memory as counters.
Thus, one potential way to understand the sucess of these networks is to revisit the theory of counter computation.
Therefore, we choose to study the abilities of real-time counter machines as formal grammars.
We first show that several variants of the counter machine converge to express the same class of formal languages.
We also prove that counter languages are closed under complement, union, intersection, and many other common set operations.
Next, we show that counter machines cannot evaluate boolean expressions, even though they can weakly validate their syntax.
This has implications for the interpretability and evaluation of neural network systems: successfully matching syntactic patterns does not guarantee that a counter-like model accurately represents underlying semantic structures.
Finally, we consider the question of whether counter languages are semilinear.
This work makes general contributions to the theory of formal languages that are of particular interest for the interpretability of recurrent neural networks.

assistant: text: We study the class of formal languages acceptable by real-time counter automata, a model of computation related to some types of recurrent neural networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Supervised Policy Update for Deep Reinforcement Learning

Paper content:
We propose a new sample-efficient methodology, called Supervised Policy Update (SPU), for deep reinforcement learning.
Starting with data generated by the current policy, SPU formulates and solves a constrained optimization problem in the non-parameterized proximal policy space.
Using supervised regression, it then converts the optimal non-parameterized policy to a parameterized policy, from which it draws new samples.
The methodology is general in that it applies to both discrete and continuous action spaces, and can handle a wide variety of proximity constraints for the non-parameterized optimization problem.
We show how the Natural Policy Gradient and Trust Region Policy Optimization (NPG/TRPO) problems, and the Proximal Policy Optimization (PPO) problem can be addressed by this methodology.
The SPU implementation is much simpler than TRPO.
In terms of sample efficiency, our extensive experiments show SPU outperforms TRPO in Mujoco simulated robotic tasks and outperforms PPO in Atari video game tasks.

assistant: text: first posing and solving the sample efficiency optimization problem in the non-parameterized policy space, and then solving a supervised regression problem to find a parameterized policy that is near the optimal non-parameterized policy.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Copula approach for hyperparameter transfer learning

Paper content:
Bayesian optimization (BO) is a popular methodology to tune the hyperparameters of expensive black-box functions.
Despite its success, standard BO focuses on a single task at a time and is not designed to leverage information from related functions, such as tuning performance metrics of the same algorithm across multiple datasets.
In this work, we introduce a novel approach to achieve transfer learning across different datasets as well as different metrics.
The main idea is to regress the mapping from hyperparameter to metric quantiles with a semi-parametric Gaussian Copula distribution, which provides robustness against different scales or outliers that can occur in different tasks.
We introduce two methods to leverage this estimation: a Thompson sampling strategy as well as a Gaussian Copula process using such quantile estimate as a prior.
We show that these strategies can combine the estimation of multiple metrics such as runtime and accuracy, steering the optimization toward cheaper hyperparameters for the same level of accuracy.
Experiments on an extensive set of hyperparameter tuning tasks demonstrate significant improvements over state-of-the-art methods.

assistant: text: We show how using semi-parametric prior estimations can speed up HPO significantly across datasets and metrics.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Variational Inference of Disentangled Latent Concepts from Unlabeled Observations

Paper content:
Disentangled representations, where the higher level data generative factors are reflected in disjoint latent dimensions, offer several benefits such as ease of deriving invariant representations, transferability to other tasks, interpretability, etc.
We consider the problem of unsupervised learning of disentangled representations from large pool of unlabeled observations, and propose a variational inference based approach to infer disentangled latent factors.
We introduce a regularizer on the expectation of the approximate posterior over observed data that encourages the disentanglement.
We also propose a new disentanglement metric which is better aligned with the qualitative disentanglement observed in the decoder's output.
We empirically observe significant improvement over existing methods in terms of both disentanglement and data likelihood (reconstruction quality).

assistant: text: We propose a variational inference based approach for encouraging the inference of disentangled latents. We also propose a new metric for quantifying disentanglement.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Chargrid-OCR: End-to-end trainable Optical Character Recognition through Semantic Segmentation and Object Detection

Paper content:
We present an end-to-end trainable approach for optical character recognition (OCR) on printed documents.
It is based on predicting a two-dimensional character grid ('chargrid') representation of a document image as a semantic segmentation task.
To identify individual character instances from the chargrid, we regard characters as objects and use object detection techniques from computer vision.
We demonstrate experimentally that our method outperforms previous state-of-the-art approaches in accuracy while being easily parallelizable on GPU (thereby being significantly faster), as well as easier to train.

assistant: text: End-to-end trainable Optical Character Recognition on printed documents; we achieve state-of-the-art results, beating Tesseract4 on benchmark datasets both in terms of accuracy and runtime, using a purely computer vision based approach.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: V4D: 4D Convonlutional Neural Networks for Video-level Representation Learning

Paper content:
Most existing 3D CNN structures for video representation learning are clip-based methods, and do not consider video-level temporal evolution of spatio-temporal features.
In this paper, we propose Video-level 4D Convolutional Neural Networks, namely V4D, to model the evolution of long-range spatio-temporal representation with 4D convolutions, as well as preserving 3D spatio-temporal representations with residual connections.
We further introduce the training and inference methods for the proposed V4D.
Extensive experiments are conducted on three video recognition benchmarks, where V4D achieves excellent results, surpassing recent 3D CNNs by a large margin.

assistant: text: A novel 4D CNN structure for video-level representation learning, surpassing  recent 3D CNNs.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Information Theoretic Co-Training

Paper content:
This paper introduces an information theoretic co-training objective for unsupervised learning.
We consider the problem of predicting the future
. Rather than predict future sensations (image pixels or sound waves) we predict ``hypotheses'' to be confirmed by future sensations
. More formally, we assume a population distribution on pairs $(x,y)$ where we can think of $x$ as a past sensation and $y$ as a future sensation
. We train both a predictor model $P_\Phi(z|x)$ and a confirmation model $P_\Psi(z|y)$ where we view $z$ as hypotheses (when predicted) or facts (when confirmed
). For a population distribution on pairs $(x,y)$ we focus on the problem of measuring the mutual information between $x$ and $
y$. By the data processing inequality this mutual information is at least as large as the mutual information between $x$ and $z$ under the distribution on triples $(x,z,y)$ defined by the confirmation model $P_\Psi(z|y)$.
The information theoretic training objective for $P_\Phi(z|x)$ and $P_\Psi(z|y)$ can be viewed as a form of co-training where we want the prediction from $x$ to match the confirmation from $y$.
We give experiments on applications to learning phonetics on the TIMIT dataset.

assistant: text: Presents an information theoretic training objective for co-training and demonstrates its power in unsupervised learning of phonetics.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Annealed Denoising score matching: learning Energy based model in high-dimensional spaces

Paper content:
Energy  based  models  outputs  unmormalized  log-probability  values  given  datasamples.
Such a estimation is essential in a variety of application problems suchas sample generation, denoising, sample restoration, outlier detection, Bayesianreasoning, and many more.
However, standard maximum likelihood training iscomputationally expensive due to the requirement of sampling model distribution.
Score matching potentially alleviates this problem, and denoising score matching(Vincent, 2011) is a particular convenient version.
However,  previous attemptsfailed to produce models capable of high quality sample synthesis.
We believethat  it  is  because  they  only  performed  denoising  score  matching  over  a  singlenoise scale.
To overcome this limitation, here we instead learn an energy functionusing all noise scales.
When sampled using Annealed Langevin dynamics andsingle step denoising jump, our model produced high-quality samples comparableto state-of-the-art techniques such as GANs, in addition to assigning likelihood totest data comparable to previous likelihood models.
Our model set a new sam-ple quality baseline in likelihood-based models.
We further demonstrate that our model learns sample distribution and generalize well on an image inpainting tasks.

assistant: text: Learned energy based model with score matching