pwc-archive/papers-with-abstracts huggingface dataset:
Sub-policy Adaptation for Hierarchical Reinforcement Learning
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Decision Making, Hierarchical Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Hierarchical Reinforcement Learning is a promising approach to long-horizon decision-making problems with sparse rewards. Unfortunately, most methods still decouple the lower-level skill acquisition process and the training of a higher level that controls the skills in a new task. Treating the skills as fixed can lead to significant sub-optimality in the transfer setting. In this work, we propose a novel algorithm to discover a set of skills, and continuously adapt them along with the higher level even when training on a new task. Our main contributions are two-fold. First, we derive a new hierarchical policy gradient, as well as an unbiased latent-dependent baseline. We introduce Hierarchical Proximal Policy Optimization (HiPPO), an on-policy method to efficiently train all levels of the hierarchy simultaneously. Second, we propose a method of training time-abstractions that improves the robustness of the obtained skills to environment changes. Code and results are available at sites.google.com/view/hippo-rl.
Personalized Student Stress Prediction with Deep Multi-Task Network
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Multi-Task Learning

With the growing popularity of wearable devices, the ability to utilize physiological data collected from these devices to predict the wearer's mental state such as mood and stress suggests great clinical applications, yet such a task is extremely challenging. In this paper, we present a general platform for personalized predictive modeling of behavioural states like students' level of stress. Through the use of Auto-encoders and Multi-Task learning we extend the prediction of stress to both sequences of passive sensor data and high-level covariates. Our model outperforms the state-of-the-art in the prediction of stress level from mobile sensor data, obtaining a 45.6\% improvement in F1 score on the StudentLife dataset.
Learning Exploration Policies for Model-Agnostic Meta-Reinforcement Learning
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Efficient Exploration, Meta Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Meta-Reinforcement learning approaches aim to develop learning procedures that can adapt quickly to a distribution of tasks with the help of a few examples. Developing efficient exploration strategies capable of finding the most useful samples becomes critical in such settings. Existing approaches to finding efficient exploration strategies add auxiliary objectives to promote exploration by the pre-update policy, however, this makes the adaptation using a few gradient steps difficult as the pre-update (exploration) and post-update (exploitation) policies are quite different. Instead, we propose to explicitly model a separate exploration policy for the task distribution. Having two different policies gives more flexibility in training the exploration policy and also makes adaptation to any specific task easier. We show that using self-supervised or supervised learning objectives for adaptation stabilizes the training process and also demonstrate the superior performance of our model compared to prior works in this domain.
A Functional Extension of Multi-Output Learning
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6

When considering simultaneously a finite number of tasks, multi-output learning enables one to account for the similarities of the tasks via appropriate regularizers. We propose a generalization of the classical setting to a continuum of tasks by using vector-valued RKHSs.
Data Enrichment: Multi-task Learning in High Dimension with Theoretical Guarantees
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Multi-Task Learning, Vocal Bursts Intensity Prediction

Given samples from a group of related regression tasks, a data-enriched model describes observations by a common and per-group individual parameters. In high-dimensional regime, each parameter has its own structure such as sparsity or group sparsity. In this paper, we consider the general form of data enrichment where data comes in a fixed but arbitrary number of tasks $G$ and any convex function, e.g., norm, can characterize the structure of both common and individual parameters. 	We propose an estimator for the high-dimensional data enriched model and investigate its statistical properties.  We delineate the sample complexity of our estimator and provide high probability non-asymptotic bound for estimation error of all parameters under a condition weaker than the state-of-the-art. We propose an iterative estimation algorithm with a geometric convergence rate. Overall, we present a first through statistical and computational analysis of inference in the data enriched model.
PAGANDA: An Adaptive Task-Independent Automatic Data Augmentation
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: BIG-bench Machine Learning, Data Augmentation, image-classification, Image Classification, Image Inpainting

Data augmentation is a useful technique to enlarge the size of the training set and prevent overfitting for different machine learning tasks when training data is scarce. However, current data augmentation techniques rely heavily on human design and domain knowledge, and existing automated approaches are yet to fully exploit the latent features in the training dataset. In this paper we propose  \textit{Parallel Adaptive GAN Data Augmentation}(PAGANDA), where the training set adaptively enriches itself with sample images automatically constructed from Generative Adversarial Networks (GANs) trained in parallel. We demonstrate by experiments that our data augmentation strategy, with little model-specific considerations, can be easily adapted to cross-domain deep learning/machine learning tasks such as image classification and image inpainting, while significantly improving model performance in both tasks. Our source code and experimental details are available at \url{https://github.com/miaojiang1987/k-folder-data-augmentation-gan/}.
Progressive Memory Banks for Incremental Domain Adaptation
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Domain Adaptation

This paper addresses the problem of incremental domain adaptation (IDA). We assume each domain comes sequentially, and that we could only access data in the current domain. The goal of IDA is  to build a unified model performing well on all the encountered domains. We propose to augment a recurrent neural network (RNN) with a directly parameterized memory bank, which is retrieved by an attention mechanism at each step of RNN transition. The memory bank provides a natural way of IDA: when adapting our model to a new domain, we progressively add new slots to the memory bank, which increases the model capacity. We learn the new memory slots and fine-tune existing parameters by back-propagation. 
Experiments show that our approach significantly outperforms naive fine-tuning and previous work on IDA, including elastic weight consolidation and the progressive neural network.  Compared with expanding hidden states, our approach is more robust for old domains, shown by both empirical and theoretical results.
Federated Optimization for Heterogeneous Networks
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Federated Learning, Multi-Task Learning

Federated learning involves training and effectively combining machine learning models from distributed partitions of data (i.e., tasks) on edge devices, and be naturally viewed as a multi- task learning problem. While Federated Averaging (FedAvg) is the leading optimization method for training non-convex models in this setting, its behavior is not well understood in realistic federated settings when the devices/tasks are statistically heterogeneous, i.e., where each device collects data in a non-identical fashion. In this work, we introduce a framework, called FedProx, to tackle statistical heterogeneity. FedProx encompasses FedAvg as a special case. We provide convergence guarantees for FedProx through a device dissimilarity assumption. Our empirical evaluation validates our theoretical analysis and demonstrates the improved robustness and stability of FedProx for learning in heterogeneous networks.
Interpretable Robust Recommender Systems with Side Information
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Recommendation Systems, regression

In this paper, we propose two methods, namely Trace-norm regression (TNR) and Stable Trace-norm Analysis (StaTNA), to improve performances of recommender systems with side information. Our trace-norm regression approach extracts low-rank latent factors underlying the side information that drives user preference under different context. Furthermore, our novel recommender framework StaTNA not only captures latent low-rank common drivers for user preferences, but also considers idiosyncratic taste for individual users. We compare performances of TNR and StaTNA on the MovieLens datasets against state-of-the-art models, and demonstrate that StaTNA and TNR in general outperforms these methods.
Tasks Without Borders: A New Approach to Online Multi-Task Learning
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Lifelong learning, Multi-Task Learning

We introduce MTLAB, a new algorithm for learning multiple related tasks with strong theoretical guarantees. Its key idea is to perform learning sequentially over the data of all tasks, without interruptions or restarts at task boundaries. Predictors for individual tasks are derived from this process by an additional online-to-batch conversion step.

By learning across task boundaries, MTLAB achieves a sublinear regret of true risks in the number of tasks. In the lifelong learning setting, this leads to an improved generalization bound that converges with the total number of samples across all observed tasks, instead of the number of examples per tasks or the number of tasks independently. At the same time, it is widely applicable: it can handle finite sets of tasks, as common in multi-task learning, as well as stochastic task sequences, as studied in lifelong learning.
The Role of Embedding Complexity in Domain-invariant Representations
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation aims to generalize the hypothesis trained in a source domain to an unlabeled target domain. One popular approach to this problem is to learn a domain-invariant representation for both domains. In this work, we study, theoretically and empirically, the explicit effect of the embedding on generalization to the target domain. In particular, the complexity of the class of embeddings affects an upper bound on the target domain's risk. This is reflected in our experiments, too.
Every Sample a Task: Pushing the Limits of Heterogeneous Models with Personalized Regression
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: BIG-bench Machine Learning, regression

When data arise from multiple latent subpopulations, machine learning frameworks typically estimate parameter values independently for each sub-population. In this paper, we propose
to overcome these limits by considering samples as tasks in a multitask learning framework.
A Meta Understanding of Meta-Learning
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Data Augmentation, Meta-Learning

Recent years have witnessed an abundance of new publications and approaches on meta-learning. This community-wide enthusiasm has sparked great insights but has also created a plethora of seemingly different frameworks, which can be hard to compare and evaluate. In this paper, we aim to provide a single principled, unifying framework that draws a close connection between meta-learning and traditional supervised learning. By treating pairs of task-specific data sets and trained models as (feature, label) samples, we can reduce many meta-learning algorithms to instances of supervised learning. This view not only unifies meta-learning into an intuitive and practical framework but also allows us to transfer insights from supervised learning directly to improve meta-learning. For example, we obtain a better understanding of generalization properties, and we can readily transfer well-understood techniques, such as model ensemble, pre-training, joint training, data augmentation, and even nearest neighbor based methods.  We provide an intuitive analogy of these methods in the context of meta-learning and show that they give rise to significant improvements in model performance.
TuckER: Tensor Factorization for Knowledge Graph Completion
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Knowledge Graph Completion, Knowledge Graphs, Link Prediction, Multi-Task Learning, Prediction

Knowledge graphs are structured representations of real world facts. However, they typically contain only a small subset of all possible facts. Link prediction is the task of inferring missing facts based on existing ones. We propose TuckER, a relatively simple yet powerful linear model based on Tucker decomposition of the binary tensor representation of knowledge graph triples. By using this particular decomposition, parameters are shared between relations, enabling multi-task learning. TuckER outperforms previous state-of-the-art models across several standard link prediction datasets.
Meta-Reinforcement Learning for Adaptive Autonomous Driving
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Autonomous Driving, Meta Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning (RL) methods achieved major advances in multiple tasks surpassing human performance. However, most of RL strategies show a certain degree of weakness and may become computationally intractable when dealing with high-dimensional and non-stationary environments. In this paper, we build a meta-reinforcement learning (MRL) method embedding an adaptive neural network (NN) controller for efficient policy iteration in changing task conditions. Our main goal is to extend RL application to the challenging task of urban autonomous driving in CARLA simulator.
Active Multitask Learning with Committees
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Transfer Learning

The cost of annotating training data has traditionally been a bottleneck for supervised learning approaches. The problem is further exacerbated when supervised learning is applied to a number of correlated tasks simultaneously since the amount of labels required scales with the number of tasks. To mitigate this concern, we propose an active multitask learning algorithm that achieves knowledge transfer between tasks. The approach forms a so-called committee for each task that jointly makes decisions and directly shares data across similar tasks. Our approach reduces the number of queries needed during training while maintaining high accuracy on test data. Empirical results on benchmark datasets show significant improvements on both accuracy and number of query requests.
Goal-conditioned Imitation Learning
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Imitation Learning, Reinforcement Learning (RL)

Designing rewards for Reinforcement Learning (RL) is challenging because it needs to convey the desired task, be efficient to optimize, and be easy to compute. The latter is particularly problematic when applying RL to robotics, where detecting whether the desired configuration is reached might require considerable supervision and instrumentation. Furthermore, we are often interested in being able to reach a wide range of configurations, hence setting up a different reward every time might be unpractical. Methods like Hindsight Experience Replay (HER) have recently shown promise to learn policies able to reach many goals, without the need of a reward. Unfortunately, without tricks like resetting to points along the trajectory, HER might take a very long time to discover how to reach certain areas of the state-space. In this work we investigate different approaches to incorporate demonstrations to drastically speed up the convergence to a policy able to reach any goal, also surpassing the performance of an agent trained with other Imitation Learning algorithms. Furthermore, our method can be used when only trajectories without expert actions are available, which can leverage kinestetic or third person demonstration.
Connections Between Optimization in Machine Learning and Adaptive Control
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: BIG-bench Machine Learning

This paper explores many immediate connections between adaptive control and machine learning, both through common update laws as well as common concepts. Adaptive control as a field has focused on mathematical rigor and guaranteed convergence. The rapid advances in machine learning on the other hand have brought about a plethora of new techniques and problems for learning. This paper elucidates many of the numerous common connections between both fields such that results from both may be leveraged together to solve new problems. In particular, a specific problem related to higher order learning is solved through insights obtained from these intersections.
Lifelong Learning via Online Leverage Score Sampling
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Continual Learning, Lifelong learning

In order to mimic the human ability of continual acquisition and transfer of knowledge across various tasks, a learning system needs the capability for life-long learning, effectively utilizing the previously acquired skills. As such, the key challenge is to transfer and generalize the knowledge learned from one task to other tasks, avoiding interference from previous knowledge and improving the overall performance. In this paper, within the continual learning paradigm, we introduce a method that effectively forgets the less useful data samples continuously across different tasks. The method uses statistical leverage score information to measure the importance of the data samples in every task and adopts frequent directions approach to enable a life-long learning property. This effectively maintains a constant training size across all tasks. We first provide some mathematical intuition for the method and then demonstrate its effectiveness with experiments on variants of MNIST and CIFAR100 datasets.
SuperTML: Domain Transfer from Computer Vision to Structured Tabular Data through Two-Dimensional Word Embedding
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Classification, text-classification, Text Classification, Word Embeddings

Structured tabular data is the most commonly used form of data in industry according to a Kaggle ML and DS Survey. Gradient Boosting Trees, Support Vector Machine, Random Forest, and Logistic Regression are typically used for classification tasks on tabular data. The recent work of Super Characters method using two-dimensional word embeddings achieved state-of-the-art results in text classification tasks, showcasing the promise of this new approach. In this paper, we propose the SuperTML method, which borrows the idea of Super Characters method and two-dimensional embeddings to address the problem of classification on tabular data. For each input of tabular data, the features are first projected into two-dimensional embeddings like an image, and then this image is fed into fine-tuned ImageNet CNN models for classification. Experimental results have shown that the proposed SuperTML method have achieved state-of-the-art results on both large and small datasets.
Enabling Continual Learning in Neural Networks with Meta Learning
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Continual Learning, Meta-Learning, Transfer Learning

Catastrophic forgetting in neural networks is one of the most well-known problems in continual learning. Previous attempts on addressing the problem focus on preventing important weights from changing. Such methods often require task boundaries to learn effectively and do not support backward transfer learning. In this paper, we propose a meta-learning algorithm which learns to reconstruct the gradients of old tasks w.r.t. the current parameters and combines these reconstructed gradients with the current gradient to enable continual learning and backward transfer learning from the current task to previous tasks. Experiments on standard continual learning benchmarks show that our algorithm can effectively prevent catastrophic forgetting and supports backward transfer learning.
System Demo for Transfer Learning from Vision to Language using Domain Specific CNN Accelerator for On-Device NLP Applications
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Sentence, Sentiment Analysis, text-classification, Text Classification, Transfer Learning

Power-efficient CNN Domain Specific Accelerator (CNN-DSA) chips are currently available for wide use in mobile devices. These chips are mainly used in computer vision applications. However, the recent work of Super Characters method for text classification and sentiment analysis tasks using two-dimensional CNN models has also achieved state-of-the-art results through the method of transfer learning from vision to text. In this paper, we implemented the text classification and sentiment analysis applications on mobile devices using CNN-DSA chips. Compact network representations using one-bit and three-bits precision for coefficients and five-bits for activations are used in the CNN-DSA chip with power consumption less than 300mW. For edge devices under memory and compute constraints, the network is further compressed by approximating the external Fully Connected (FC) layers within the CNN-DSA chip. At the workshop, we have two system demonstrations for NLP tasks. The first demo classifies the input English Wikipedia sentence into one of the 14 classes. The second demo classifies the Chinese online-shopping review into positive or negative.
SuperChat: Dialogue Generation by Transfer Learning from Vision to Language using Two-dimensional Word Embedding and Pretrained ImageNet CNN Models
Authors: Anonymous
Date: 2019-05-16 00:00:00
Proceeding: ICML Workshop AMTL 2019 6
Tasks: Dialogue Generation, text-classification, Text Classification, Transfer Learning

The recent work of Super Characters method using two-dimensional word embedding achieved state-of-the-art results in text classification tasks, showcasing the promise of this new approach. This paper borrows the idea of Super Characters method and two-dimensional embedding, and proposes a method of generating conversational response for open domain dialogues. The experimental results on a public dataset shows that the proposed SuperChat method generates high quality responses. An interactive demo is ready to show at the workshop. And code will be available at github soon.