pwc-archive/papers-with-abstracts huggingface dataset:
Data Augmentation for Rumor Detection Using Context-Sensitive Neural Language Model With Large-Scale Credibility Corpus
Authors: Sooji Han, Jie Gao, Fabio Ciravegna
Date: 2019-03-24 00:00:00
Proceeding: ICLR Workshop LLD 2019
Tasks: Data Augmentation, Language Modeling, Language Modelling

In this paper, we address the challenge of limited labeled data and class imbalance problem for machine learning-based rumor detection on social media. We present an offline data augmentation method based on semantic relatedness for rumor detection. To this end, unlabeled social media data is exploited to augment limited labeled data. A context-aware neural language model and a large credibility-focused Twitter corpus are employed to learn effective representations of rumor tweets for semantic relatedness measurement. A language model fine-tuned with the a large domain-specific corpus shows a dramatic improvement on training data augmentation for rumor detection over pretrained language models. We conduct experiments on six different real-world events based on five publicly available data sets and one augmented data set. Our experiments show that the proposed method allows us to generate a larger training data with reasonable quality via weak supervision. We present preliminary results achieved using a state-of-the-art neural network model with augmented data for rumor detection.
Enhancing experimental signals in single-cell RNA-sequencing data using graph signal processing
Authors: Daniel B. Burkhardt, Jay S. Stanley III, Ana Luisa Perdigoto, Scott A. Gigante, Kevan C. Herold, Guy Wolf, Antonio J. Giraldez, David van Dijk, Smita Krishnaswamy
Date: 2019-03-24 00:00:00
Proceeding: ICLR Workshop LLD 2019

Single-cell RNA-sequencing (scRNA-seq) is a powerful tool for analyzing biological systems. However, due to biological and technical noise, quantifying the effects of multiple experimental conditions presents an analytical challenge. To overcome this challenge, we developed MELD: Manifold Enhancement of Latent Dimensions. MELD leverages tools from graph signal processing to learn a latent dimension within the data scoring the prototypicality of each datapoint with respect to experimental or control conditions. We call this dimension the Enhanced Experimental Signal (EES). MELD learns the EES by filtering the noisy categorical experimental label in the graph frequency domain to recover a smooth signal with continuous values. This method can be used to identify signature genes that vary between conditions and identify which cell types are most affected by a given perturbation. We demonstrate the advantages of MELD analysis in two biological datasets, including T-cell activation in response to antibody-coated beads and treatment of human pancreatic islet cells with interferon gamma.
Automatic Labeling of Data for Transfer Learning
Authors: Parijat Dube, Bishwaranjan Bhattacharjee, Siyu Huo, Patrick Watson, John Kender, Brian Belgodere
Date: 2019-03-24 00:00:00
Tasks: Transfer Learning

Transfer learning uses trained weights from a source model as the initial weightsfor the training of a target dataset.  A well chosen source with a large numberof labeled data leads to significant improvement in accuracy.  We demonstrate atechnique that automatically labels large unlabeled datasets so that they can trainsource models for transfer learning. We experimentally evaluate this method, usinga baseline dataset of human-annotated ImageNet1K labels, against five variationsof this technique.  We show that the performance of these automatically trainedmodels come within 17% of baseline on average.
Learning To Avoid Negative Transfer in Few Shot Transfer Learning
Authors: James O' Neill
Date: 2019-03-24 00:00:00
Tasks: Few-Shot Learning, Natural Language Inference, Natural Language Understanding, Transfer Learning

Many tasks in natural language understanding require learning relationships between two sequences for various tasks such as natural language inference, paraphrasing and entailment. These aforementioned tasks are similar in nature, yet they are often modeled individually. Knowledge transfer can be effective for closely related tasks, which is usually carried out using parameter transfer in neural networks. However, transferring all parameters, some of which irrelevant for a target task, can lead to sub-optimal results and can have a negative effect on performance, referred to as \textit{negative} transfer. 

Hence, this paper focuses on the transferability of both instances and parameters across natural language understanding tasks by proposing an ensemble-based transfer learning method in the context of few-shot learning.

Our main contribution is a method for mitigating negative transfer across tasks when using neural networks, which involves dynamically bagging small recurrent neural networks trained on different subsets of the source task/s. We present a straightforward yet novel approach for incorporating these networks to a target task for few-shot learning by using a decaying parameter chosen according to the slope changes of a smoothed spline error curve at sub-intervals during training.

Our proposed method show improvements over hard and soft parameter sharing transfer methods in the few-shot learning case and shows competitive performance against models that are trained given full supervision on the target task, from only few examples.
Multi-Class Few Shot Learning Task and Controllable Environment
Authors: Dmitriy Serdyuk, Negar Rostamzadeh, Pedro Oliveira Pinheiro, Boris Oreshkin, Yoshua Bengio
Date: 2019-03-24 00:00:00
Tasks: Classification, Few-Shot Learning

Deep learning approaches usually require a large amount of labeled data to generalize. However, humans can learn a new concept only by a few samples. One of the high cogntition human capablities is to learn several concepts at the same time. In this paper, we address the task of classifying multiple objects by seeing only a few samples from each category. To the best of authors' knowledge, there is no dataset specially designed for few-shot multiclass classification. We design a task of mutli-object few class classification and an environment for easy creating controllable datasets for this task. We demonstrate that the proposed dataset is sound using a method which is an extension of prototypical networks.
Efficient Receptive Field Learning by Dynamic Gaussian Structure
Authors: Evan Shelhamer, Dequan Wang, Trevor Darrell
Date: 2019-03-24 00:00:00
Proceeding: ICLR Workshop LLD 2019
Tasks: Form, Representation Learning

The visual world is vast and varied, but its variations divide into structured and unstructured factors. Structured factors, such as scale and orientation, admit clear theories and efficient representation design. Unstructured factors, such as what it is that makes a cat look like a cat, are too complicated to model analytically, and so require free-form representation learning. We compose structured Gaussian filters and free-form filters, optimized end-to-end, to factorize the representation for efficient yet general learning. Our experiments on dynamic structure, in which the structured filters vary with the input, equal the accuracy of dynamic inference with more degrees of freedom while improving efficiency.

(Please see https://arxiv.org/abs/1904.11487 for the full edition.)
Data for free: Fewer-shot algorithm learning with parametricity data augmentation
Authors: Owen Lewis, Katherine Hermann
Date: 2019-03-24 00:00:00
Proceeding: ICLR Workshop LLD 2019
Tasks: Data Augmentation

We address the problem of teaching an RNN to approximate list-processing algorithms given a small number of input-output training examples. Our approach is to generalize the idea of parametricity from programming language theory to formulate a semantic property that distinguishes common algorithms from arbitrary non-algorithmic functions. This characterization leads naturally to a learned data augmentation scheme that encourages RNNs to learn algorithmic behavior and enables small-sample learning in a variety of list-processing tasks.
A Pseudo-Label Method for Coarse-to-Fine Multi-Label Learning with Limited Supervision
Authors: Cheng-Yu Hsieh, Miao Xu, Gang Niu, Hsuan-Tien Lin, Masashi Sugiyama
Date: 2019-03-25 00:00:00
Proceeding: ICLR Workshop LLD 2019
Tasks: Meta-Learning, Multi-Label Learning, Pseudo Label

The goal of multi-label learning (MLL) is to associate a given instance with its relevant labels from a set of concepts. Previous works of MLL mainly focused on the setting where the concept set is assumed to be fixed, while many real-world applications require introducing new concepts into the set to meet new demands. One common need is to refine the original coarse concepts and split them into finer-grained ones, where the refinement process typically begins with limited labeled data for the finer-grained concepts. To address the need, we propose a special weakly supervised MLL problem that not only focuses on the situation of limited fine-grained supervision but also leverages the hierarchical relationship between the coarse concepts and the fine-grained ones. The problem can be reduced to a multi-label version of negative-unlabeled learning problem using the hierarchical relationship. We tackle the reduced problem with a meta-learning approach that learns to assign pseudo-labels to the unlabeled entries. Experimental results demonstrate that our proposed method is able to assign accurate pseudo-labels, and in turn achieves superior classification performance when compared with other existing methods.
Parallel Recurrent Data Augmentation for GAN training with Limited and Diverse Data
Authors: Boli Fang, Miao Jiang
Date: 2019-03-25 00:00:00
Tasks: Data Augmentation, Image Generation

The need for large amounts of training image data with clearly defined features is a major obstacle to applying generative adversarial networks(GAN) on image generation where training data is limited but diverse, since insufficient latent feature representation in the already scarce data often leads to instability and mode collapse during GAN training. To overcome the hurdle of limited data when applying GAN to limited datasets, we propose in this paper the strategy of \textit{parallel recurrent data augmentation}, where the GAN model progressively enriches its training set with sample images constructed from GANs trained in parallel at consecutive training epochs. Experiments on a variety of small yet diverse datasets demonstrate that our method, with little model-specific considerations, produces images of better quality as compared to the images generated  without such strategy. The source code and generated images of this paper will be made public after review.
Improving Sample Complexity with Observational Supervision
Authors: Khaled Saab, Jared Dunnmon, Alexander Ratner, Daniel Rubin, Christopher Re
Date: 2019-03-25 00:00:00
Proceeding: ICLR Workshop LLD 2019
Tasks: image-classification, Image Classification, Medical Image Classification

Supervised machine learning models for high-value computer vision applications such as medical image classification often require large datasets labeled by domain experts, which are slow to collect, expensive to maintain, and static with respect to changes in the data distribution. In this context, we assess the utility of observational supervision, where we take advantage of passively-collected signals such as eye tracking or “gaze” data, to reduce the amount of hand-labeled data needed for model training. Specifically, we leverage gaze information to directly supervise a visual attention layer by penalizing disagreement between the spatial regions the human labeler looked at the longest and those that most heavily influence model output. We present evidence that constraining the model in this way can reduce the number of labeled examples required to achieve a given performance level by as much as 50%, and that gaze information is most helpful on more difficult tasks.
Weak Supervision for Time Series: Wearable Sensor Classification with Limited Labeled Data
Authors: Saelig Khattar, Hannah O’Day, Paroma Varma, Jason Fries, Jen Hicks, Scott Delp, Helen Bronte-Stewart, Chris Re
Date: 2019-03-25 00:00:00
Tasks: Time Series, Time Series Analysis

Using modern deep learning models to make predictions on time series data from wearable sensors generally requires large amounts of labeled data. However, labeling these large datasets can be both cumbersome and costly. In this paper, we apply weak supervision to time series data, and programmatically label a dataset from sensors worn by patients with Parkinson's. We then built a LSTM model that predicts when these patients exhibit clinically relevant freezing behavior (inability to make effective forward stepping). We show that (1) when our model is trained using patient-specific data (prior sensor sessions), we come within 9% AUROC of a model trained using hand-labeled data and (2) when we assume no prior observations of subjects, our weakly supervised model matched performance with hand-labeled data. These results demonstrate that weak supervision may help reduce the need to painstakingly hand label time series training data.
Few-Shot Regression via Learned Basis Functions
Authors: Yi Loo, Swee Kiat Lim, Gemma Roig, Ngai-Man Cheung
Date: 2019-03-25 00:00:00
Proceeding: ICLR Workshop LLD 2019
Tasks: Few-Shot Learning, Meta-Learning, regression

The recent rise in popularity of few-shot learning algorithms has enabled models to quickly adapt to new tasks based on only a few training samples. Previous few-shot learning works have mainly focused on classification and reinforcement learning. 
In this paper, we propose a few-shot meta-learning system that focuses exclusively on regression tasks. Our model is based on the idea that the degree of freedom of the unknown function can be significantly reduced if it is represented as a linear combination of a set of appropriate basis functions. This enables a few labelled samples to approximate the function. We design a Feature Extractor network to encode basis functions for a task distribution, and a  Weights Generator to generate the weight vector for a novel task. We show that our model outperforms the current state of the art meta-learning methods in various regression tasks.
Interactions between Representation Learning and Supervision
Authors: Valliappa Chockalingam
Date: 2019-03-25 00:00:00
Tasks: Continual Learning, Dimensionality Reduction, Permuted-MNIST, Reinforcement Learning (RL), Representation Learning

Representation learning is one of the fundamental problems of machine learning. On its own, this problem can be cast as an unsupervised dimensionality reduction problem. However, representation learning is often also used as an implicit step in supervised learning (SL) or reinforcement learning (RL) problems. In this paper, we study the possible "interference" supervision, commonly provided through a loss function in SL or a reward function in RL, might have on learning representations, through the lens of learning from limited data and continual learning. Particularly, in connectionist networks, we often face the problem of catastrophic interference whereby changes in the data distribution cause networks to fail to remember previously learned information and learning representations can be done without labeled data. A primary running hypothesis is that representations learned using unsupervised learning are more robust to changes in the data distribution as compared to the intermediate representations learned when using supervision because supervision interferes with otherwise "unconstrained" representation learning objectives. To empirically test hypotheses, we perform experiments using a standard dataset for continual learning, permuted MNIST. Additionally, through a heuristic quantifying the amount of change in the data distribution, we verify that the results are statistically significant.
Deep Generative Inpainting with Comparative Sample Augmentation
Authors: Boli Fang, Miao Jiang, Jerry Shen, Bjord Stenger
Date: 2019-03-25 00:00:00
Tasks: Diversity, Image Inpainting

Recent advancements in deep learning techniques such as Convolutional Neural Networks(CNN) and Generative Adversarial Networks(GAN) have achieved breakthroughs in the problem of semantic image inpainting, the task of reconstructing missing pixels in given images. While much more effective than conventional approaches, deep learning models require large datasets and great computational resources for training, and inpainting quality varies considerably when training data vary in size and diversity. To address these problems, we present in this paper a inpainting strategy of \textit{Comparative Sample Augmentation}, which enhances the quality of training set by filtering out irrelevant images and constructing additional images using information about the surrounding regions of the images to be inpainted. Experiments on multiple datasets demonstrate that our method extends the applicability of deep inpainting models to training sets with varying sizes, while maintaining inpainting quality as measured by qualitative and quantitative metrics for a large class of deep models, with little need for model-specific consideration.
A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
Authors: Anonymous
Date: 2019-03-25 00:00:00
Tasks: Learning with noisy labels

Recently deep neural networks have shown their capacity to memorize training data, even with noisy labels, which hurts generalization performance. To mitigate this issue, we propose a simple but effective method that is robust to noisy labels, even with severe noise.  Our objective involves a variance regularization term that implicitly penalizes the Jacobian norm of the neural network on the whole training set (including the noisy-labeled data), which encourages generalization and prevents overfitting to the corrupted labels. Experiments on noisy benchmarks demonstrate that our approach achieves state-of-the-art performance with a high tolerance to severe noise.
Adaptive Masked Weight Imprinting for Few-Shot Segmentation
Authors: Mennatullah Siam, Boris Oreshkin
Date: 2019-02-25 00:00:00
Proceeding: ICLR Workshop LLD 2019
Tasks: Continual Learning, Semantic Segmentation

Deep learning has mainly thrived by training on large-scale datasets. However, for continual learning in applications such as robotics, it is critical to incrementally update its model in a sample efficient manner. We propose a novel method that constructs the new class weights from few labelled samples in the support set, while updating the previously learned classes. Inspiring from the work on adaptive correlation filters, an adaptive masked imprinted weights method is proposed. It utilizes a masked average pooling layer on the output embeddings and acts as a positive proxy for that class. It is then used to adaptively update the 1x1 convolutional filters that are responsible for the final classification. Our proposed method is evaluated on PASCAL-5i dataset and outperforms the state of the art in the 5-shot semantic segmentation. Unlike previous methods, our proposed approach does not require a second branch to estimate parameters or prototypes, and it enables the adaptation of previously learned weights. We further propose a novel setup for evaluating incremental object segmentation which we term as incremental PASCAL (iPASCAL), where our adaptation method has shown to outperform the baseline method.
Reference-based Variational Autoencoders
Authors: Adrià Ruiz, Oriol Martinez, Xavier Binefa, Jakob Verbeek
Date: 2019-03-08 00:00:00
Proceeding: ICLR Workshop LLD 2019
Tasks: Attribute, Conditional Image Generation, Image Generation

Learning disentangled representations from visual data, where different high-level generative factors are independently encoded, is of importance for many computer vision tasks. Solving this problem, however, typically requires to explicitly label all the  factors of interest in training images. To alleviate the annotation cost, we introduce a learning setting which we refer to as \textit{reference-based disentangling}. Given a pool of unlabelled images, the goal is to learn a representation where a set of target factors are disentangled from others. The only supervision comes from an auxiliary \textit{reference set} containing  images where the factors of interest are constant. To address this problem, we propose reference-based variational autoencoders, a novel deep generative model designed to exploit the weak-supervision provided by the reference set. By addressing tasks such as feature learning, conditional image generation or attribute transfer, we validate the ability of the proposed model to learn disentangled representations from this minimal form of supervision.
Regularity Normalization: Constraining Implicit Space with Minimum Description Length
Authors: Baihan Lin
Date: 2019-03-11 00:00:00
Tasks: Model Selection

Inspired by the adaptation phenomenon of biological neuronal firing, we propose regularity normalization: a reparameterization of the activation in the neural network that take into account the statistical regularity in the implicit space. By considering the neural network optimization process as a model selection problem, the implicit space is constrained by the normalizing factor, the minimum description length of the optimal universal code. We introduce an incremental version of computing this universal code as normalized maximum likelihood and demonstrated its flexibility to include data prior such as top-down attention and other oracle information and its compatibility to be incorporated into batch normalization and layer normalization. The preliminary results showed that the proposed method outperforms existing normalization methods in tackling the limited and imbalanced data from a non-stationary distribution benchmarked on computer vision task. As an unsupervised attention mechanism given input data, this biologically plausible normalization has the potential to deal with other complicated real-world scenarios as well as reinforcement learning setting where the rewards are sparse and non-uniform. Further research is proposed to discover these scenarios and explore the behaviors among different variants.
WEAKLY SEMI-SUPERVISED NEURAL TOPIC MODELS
Authors: Ian Gemp, Ramesh Nallapati, Ran Ding, Feng Nan, Bing Xiang
Date: 2019-03-13 00:00:00
Proceeding: ICLR Workshop LLD 2019
Tasks: Topic Models

We consider the problem of topic modeling in a weakly semi-supervised setting. In this scenario, we assume that the user knows a priori a subset of the topics she wants the model to learn and is able to provide a few exemplar documents for those topics. In addition, while each document may typically consist of multiple topics, we do not assume that the user will identify all its topics exhaustively.
      
Recent state-of-the-art topic models such as NVDM, referred to herein as Neural Topic Models (NTMs), fall under the variational autoencoder framework. We extend NTMs to the weakly semi-supervised setting by using informative priors in the training objective. After analyzing the effect of informative priors, we propose a simple modification of the NVDM model using a logit-normal posterior that we show achieves better alignment to user-desired topics versus other NTM models.