pwc-archive/papers-with-abstracts huggingface dataset:
Evaluating Robustness of Cooperative MARL
Authors: Nhan Pham, Lam M. Nguyen, Jie Chen, Thanh Lam Hoang, Subhro Das, Tsui-Wei Weng
Date: 2021-09-29 00:00:00
Tasks: continuous-control, Continuous Control, MuJoCo, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

In recent years, a proliferation of methods were developed for multi-agent reinforcement learning (MARL). In this paper, we focus on evaluating the robustness of MARL agents in continuous control tasks. In particular, we propose the first model-based approach to perform adversarial attacks for cooperative MARL. We design effective attacks to degrade the MARL agent's performance by adversarially perturbing the states of agent(s) and solving an optimization problem. In addition, we also developed several strategies to select the most vulnerable agents that help to further decrease the team reward of MARL. Extensive numerical experiments on multi-agent Mujoco tasks verify the effectiveness of our proposed approach.
Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining
Authors: Lu Miao, Xiaolong Luo, Tianlong Chen, Wuyang Chen, Dong Liu, Zhangyang Wang
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4

We present a novel framework to train a large deep neural network (DNN) for only $\textit{once}$, which can then be pruned to $\textit{any sparsity ratio}$ to preserve competitive accuracy $\textit{without any re-training}$. Conventional methods often require (iterative) pruning followed by re-training, which not only incurs large overhead beyond the original DNN training but also can be sensitive to retraining hyperparameters. Our core idea is to re-cast the DNN training as an explicit $\textit{pruning-aware}$ process: that is formulated with an auxiliary $K$-sparse polytope constraint, to encourage network weights to lie in a convex hull spanned by $K$-sparse vectors, potentially resulting in more sparse weight matrices. We then leverage a stochastic Frank-Wolfe (SFW) algorithm to solve this new constrained optimization, which naturally leads to sparse weight updates each time. We further note an overlooked fact that existing DNN initializations were derived to enhance SGD training (e.g., avoid gradient explosion or collapse), but was unaligned with the challenges of training with SFW. We hence also present the first learning-based initialization scheme specifically for boosting SFW-based DNN training. Experiments on CIFAR-10 and Tiny-ImageNet datasets demonstrate that our new framework named $\textbf{SFW-pruning}$ consistently achieves the state-of-the-art performance on various benchmark DNNs over a wide range of pruning ratios. Moreover, SFW-pruning only needs to train once on the same model and dataset, for obtaining arbitrary ratios, while requiring neither iterative pruning nor retraining. All codes will be released to the public.
Text Style Transfer with Confounders
Authors: Tianxiao Shen, Regina Barzilay, Tommi S. Jaakkola
Date: 2021-09-29 00:00:00
Tasks: Style Transfer, Text Style Transfer

Existing methods for style transfer operate either with paired sentences or distributionally matched corpora which differ only in the desired style. In this paper, we relax this restriction and consider data sources with additional confounding differences, from which the desired style needs to be inferred. Specifically, we first learn an invariant style classifier that takes out nuisance variation, and then introduce an orthogonal classifier that highlights the confounding cues. The resulting pair of classifiers guide us to transfer text in the specified direction, creating sentences of the type not seen during training. Experiments show that using positive and negative review datasets from different categories, we can successfully transfer the sentiment without changing the category.
Distributional Reinforcement Learning with Monotonic Splines
Authors: Yudong Luo, Guiliang Liu, Haonan Duan, Oliver Schulte, Pascal Poupart
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Distributional Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Distributional Reinforcement Learning (RL) differs from traditional RL by estimating the distribution over returns to capture the intrinsic uncertainty of MDPs. One key challenge in distributional RL lies in how to parameterize the quantile function when minimizing the Wasserstein metric of temporal differences. Existing algorithms use a step functions or piecewise linear functions. In this paper, we propose to learn smooth continuous quantile functions represented by monotonic rational-quadratic splines, which also naturally solve the quantile crossing problem. Experiments in stochastic environments show that a dense estimation for quantile functions enhances distributional RL in terms of faster empirical convergence and higher rewards in most cases.
Comparing Distributions by Measuring Differences that Affect Decision Making
Authors: Shengjia Zhao, Abhishek Sinha, Yutong He, Aidan Perreault, Jiaming Song, Stefano Ermon
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Decision Making

Measuring the discrepancy between two probability distributions is a fundamental problem in machine learning and statistics. We propose a new class of discrepancies based on the optimal loss for a decision task -- two distributions are different if the optimal decision loss is higher on their mixture than on each individual distribution. By suitably choosing the decision task, this generalizes the Jensen-Shannon divergence and the maximum mean discrepancy family. We apply our approach to two-sample tests, and on various benchmarks, we achieve superior test power compared to competing methods. In addition, a modeler can directly specify their preferences when comparing distributions through the decision loss. We apply this property to understanding the effects of climate change on different social and economic activities, evaluating sample quality, and selecting features targeting different decision tasks.
Approximate Bijective Correspondence for isolating factors of variation
Authors: Kieran A Murphy, Varun Jampani, Srikumar Ramalingam, Ameesh Makadia
Date: 2021-09-29 00:00:00
Tasks: Contrastive Learning, Data Augmentation, Pose Transfer

Representational learning forms the backbone of most deep learning applications, and the value of a learned representation is intimately tied to its information content regarding different factors of variation. Finding good representations depends on the nature of supervision and the learning algorithm. We propose a novel algorithm that relies on a weak form of supervision where the data is partitioned into sets according to certain \textit{inactive} factors of variation. Our key insight is that by seeking approximate correspondence between elements of different sets, we learn strong representations that exclude the inactive factors of variation and isolate the \textit{active} factors which vary within all sets. Importantly, the information isolated is complementary to that of most other contrastive learning approaches, which isolate the inactive factors of variation. We demonstrate that the method can work in a semi-supervised scenario, and that a portion of the unsupervised data can belong to a different domain entirely. Further control over the content of the learned representations is possible by folding in data augmentation to suppress nuisance factors. We outperform competing baselines on the challenging problem of synthetic-to-real object pose transfer.
Modeling Variable Space with Residual Tensor Networks for Multivariate Time Series
Authors: Jing Zhang, Peng Zhang, Yupeng He, Siwei Rao, Jun Wang, Guangjian Tian
Date: 2021-09-29 00:00:00
Tasks: Multivariate Time Series Forecasting, Tensor Networks, Time Series, Time Series Analysis, Time Series Forecasting, Translation

Multivariate time series involve a series of valuable applications in the real world, and the basic premise of which is that multiple variables are interdependent. However, the relationship between variables in the latent space is dynamic and complex, and as the time window increases, the size of the space also increases exponentially.  For fully exploiting the dependencies in the variable space, we propose Modeling Variable Space with Residual Tensor Networks (MVSRTN) for multivariate time series.  In this framework, we derive the mathematical representation of the variable space, and then use a tensor network based on the idea of low-rank approximation to model the variable space. The tensor components are shared to ensure the translation invariance of the network. In order to improve the ability to model long-term sequences, we propose an N-order residual connection approach and couple it to the space-approximated tensor network. Moreover, the series-variable encoder is designed to improve the quality of the variable space, and we use the skip-connection layer to achieve the dissemination of information such as scale. Experimental results verify the effectiveness of our proposed method on four multivariate time series forecasting benchmark datasets.
Image BERT Pre-training with Online Tokenizer
Authors: Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, Tao Kong
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: image-classification, Image Classification, Instance Segmentation, Language Modeling, Language Modelling, Masked Language Modeling, object-detection, Object Detection, Semantic Segmentation

The success of language Transformers is primarily attributed to the pretext task of masked language modeling (MLM), where texts are first tokenized into semantically meaningful pieces.
In this work, we study masked image modeling (MIM) and indicate the necessity and challenges of using a semantically meaningful visual tokenizer.
We present a self-supervised framework iBOT that can perform masked prediction with an online tokenizer. 
Specifically, we perform self-distillation on masked patch tokens and take the teacher network as the online tokenizer, along with self-distillation on the class token to acquire visual semantics.
The online tokenizer is jointly learnable with the MIM objective and dispenses with a multi-stage training pipeline where the tokenizer needs to be pre-trained beforehand.
We show the prominence of iBOT by achieving a 79.5% linear probing accuracy and an 83.8% fine-tuning accuracy with ViT-B/16 on ImageNet-1K.
Beyond the state-of-the-art image classification results, we underline emerging local semantic patterns, which helps the models to obtain strong robustness against common corruptions and achieve leading results on dense downstream tasks, e.g., object detection, instance segmentation, and semantic segmentation.
Fairness-aware Federated Learning
Authors: Zhuozhuo Tu, Zhiqiang Xu, Tairan Huang, DaCheng Tao, Ping Li
Date: 2021-09-29 00:00:00
Tasks: Fairness, Federated Learning, Generalization Bounds

Federated Learning is a machine learning technique where a network of clients collaborates with a server to learn a centralized model while keeping data localized. In such a setting, naively minimizing an aggregate loss may introduce bias and disadvantage model performance on certain clients. To address this issue, we propose a new federated learning framework called FAFL in which the goal is to minimize the worst-case weighted client losses over an uncertainty set. By deriving a variational representation, we show that this framework is a fairness-aware objective and can be easily optimized by solving a joint minimization problem over the model parameters and a dual variable. We then propose an optimization algorithm to solve FAFL which can be efficiently implemented in a federated setting and provide convergence guarantees. We further prove generalization bounds for learning with this objective. Experiments on real-world datasets demonstrate the effectiveness of our framework in achieving both accuracy and fairness.
Topic Aware Neural Language Model: Domain Adaptation of Unconditional Text Generation Models
Authors: Noriaki Kawamae
Date: 2021-09-29 00:00:00
Tasks: Domain Adaptation, Language Modeling, Language Modelling, Text Generation, valid

Our goal is to adapt pre-trained neural language models (NLMs) to the unconditional text generation task within the target domain.
Because many Transformer based NLMs are trained on more massive and heterogeneous corpora than this target domain,
the difference between these corpora and the target domain raises the question of whether these NLMs can provide their benefits to this task even after the fine-tuning.
To tackle these problems, our approach focuses on topics to bridge the semantic gap between these corpora and the target domain corpus,
and relates them at a topic level.
That is, this approach injects topics into these NLMs and trains them via topics behind these dependencies over segments,
introducing both topic alignment (TA) and training tasks (TDM and TEM),
while previous Transformer based NLMs are better at learning from the predefined segment length such as the context.
Experiments show that this approach contributes to resolve the imbalance between these corpora,
and can tailor previous pre-trained NLMs to generate coherent and semantically valid text reflecting a given small fine-tuning corpus.
Learning to Learn across Diverse Data Biases in Deep Face Recognition
Authors: Chang Liu, Xiang Yu, Yi-Hsuan Tsai, Ramin Moslemi, Masoud Faraki, Manmohan Chandraker, Yun Fu
Date: 2021-09-29 00:00:00
Proceeding: CVPR 2022 1
Tasks: Face Recognition, Meta-Learning

Convolutional Neural Networks have achieved remarkable success in face recognition, in part due to the abundant availability of data. However, the data used for training CNNs is often imbalanced. Prior works largely focus on the long-tailed nature of face datasets with respect to the number of instances per identity. In this paper, we show that besides the imbalanced class volume distribution, other variations such as ethnicity, head pose, occlusion and blur can also significantly affect accuracy. To address the problem, we propose a sample level weighting approach called Multi-variation Cosine Margin (MvCoM) which orthogonally enhances the conventional cosine loss function to incorporate the importance of training samples. Further, we leverage a learning to learn approach, guided by a held-out meta learning set and use an additive modeling to predict the MvCoM. Extensive experiments on challenging face recognition benchmarks demonstrate the advantages of our method in jointly handling imbalances due to multiple variations.
Measuring the Effectiveness of Self-Supervised Learning using Calibrated Learning Curves
Authors: Andrei Atanov, Shijian Xu, Onur Beker, Andrey Filatov, Amir Zamir
Date: 2021-09-29 00:00:00
Tasks: image-classification, Image Classification, Self-Supervised Learning, Transfer Learning

Self-supervised learning has witnessed remarkable progress in recent years, in particular with the introduction of augmentation-based contrastive methods. While a number of large-scale empirical studies on the performance of self-supervised pre-training have been conducted, there isn't yet an agreed upon set of control baselines, evaluation practices, and metrics to report. We identify this as an important angle of investigation and propose an evaluation standard that aims to quantify and communicate transfer learning performance in an informative yet accessible setup. This is done by baking in a number of key control baselines in the evaluation method, particularly the blind guess (quantifying the dataset bias), the scratch model (quantifying the architectural contribution), and the gold standard (quantifying the upper-bound). We further provide a number of experiments to demonstrate how the proposed evaluation can be employed in empirical studies of basic questions -- for example, whether the effectiveness of existing self-supervised learning methods is skewed towards image classification versus other tasks, such as dense pixel-wise predictions.
Learning Pseudometric-based Action Representations for Offline Reinforcement Learning
Authors: Pengjie Gu, Mengchen Zhao, Chen Chen, Dong Li, Jianye Hao, Bo An
Date: 2021-09-29 00:00:00
Tasks: Offline RL, Recommendation Systems, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Relation, Representation Learning

Offline reinforcement learning is a promising approach for practical applications since it does not require interactions with real-world environments. However, existing offline RL methods only work well in environments with continuous or small discrete action spaces. In environments with large and discrete action spaces, such as recommender systems and dialogue systems, the performance of existing methods decreases drastically because they suffer from inaccurate value estimation for a large proportion of o.o.d. actions. While recent works have demonstrated that online RL benefits from incorporating semantic information in action representations, unfortunately, they fail to learn reasonable relative distances between action representations, which is key to offline RL to reduce the influence of out-of-distribution (o.o.d.) actions. This paper proposes an action representation learning framework for offline RL based on a pseudometric, which measures both the behavioral relation and the data-distributional relation between actions.  We provide theoretical analysis on the continuity and the bounds of the expected Q-values using the learned action representations. Experimental results show that our methods significantly improve the performance of two typical offline RL methods in environments with large and discrete action spaces.
New Definitions and Evaluations for Saliency Methods: Staying Intrinsic and Sound
Authors: Arushi Gupta, Nikunj Saunshi, Dingli Yu, Kaifeng Lyu, Sanjeev Arora
Date: 2021-09-29 00:00:00

  Saliency methods seek to provide human-interpretable explanations for the output of machine learning model on a given input. A plethora of saliency methods exist, as well as an extensive literature on their justifications/criticisms/evaluations. This paper focuses on heat maps based saliency methods that often provide explanations that look best to humans. It tries to introduce methods and evaluations for masked-based saliency methods that are {\em intrinsic} --- use just the training dataset and the trained net, and do not use separately trained nets, distractor distributions, human evaluations or annotations. Since a mask can be seen as a "certificate" justifying the net's answer, we introduce notions of {\em completeness} and {\em soundness} (the latter being the new contribution) motivated by logical proof systems. These notions allow a new evaluation of  saliency methods, that experimentally provides a novel and stronger justification for several heuristic tricks in the field (T.V. regularization, upscaling).
One for Many: an Instagram inspired black-box adversarial attack
Authors: Alina Elena Baia, Alfredo Milani, Valentina Poggioni
Date: 2021-09-29 00:00:00
Tasks: Adversarial Attack

It is well known that deep learning models are susceptible to adversarial attacks. To produce more robust and effective attacks, we propose a nested evolutionary algorithm able to produce multi-network (decision-based) black-box adversarial attacks based on Instagram inspired image filters. Due to the multi-network training, the system reaches a high transferability rate of attacks and, due to the composition of image filters, it is able to bypass standard detection mechanisms. Moreover, this kind of attack is semantically robust: our filter composition cannot be distinguished from any other filter composition used extensively every day to enhance images; this raises new security issues and challenges for real-world systems.  Experimental results demonstrate that the method is also effective against
ensemble-adversarially trained models and it has a low cost in terms of queries to the victim model.
H-Entropy Search: Generalizing Bayesian Optimization with a Decision-theoretic Uncertainty Measure
Authors: Willie Neiswanger, Lantao Yu, Shengjia Zhao, Chenlin Meng, Stefano Ermon
Date: 2021-09-29 00:00:00
Tasks: Bayesian Optimization

Bayesian optimization (BO) is a popular method for efficiently inferring optima of an expensive black-box function via a sequence of queries. Existing information-theoretic BO procedures aim to make queries that most reduce the uncertainty about optima, where the uncertainty is captured by Shannon entropy. However, an optimal measure of uncertainty would, ideally, factor in how we intend to use the inferred quantity in some downstream procedure. In this paper, we instead consider the H-entropy, a generalization of Shannon entropy from work in statistical decision theory (DeGroot, 1962; Rao, 1984), which contains a broad class of uncertainty measures parameterized by a problem-specific loss function corresponding to a downstream task. We first show that special cases of the H-entropy lead to popular acquisition functions used in BO procedures such as knowledge gradient, expected improvement, and entropy search. We then show how alternative choices for the loss yield a flexible family of acquisition functions for a variety of specialized optimization tasks, including variants of top-k estimation, level set estimation, and multi-valued search. For special cases of the loss and design space, we develop gradient-based methods to efficiently optimize our proposed family of acquisition functions, and demonstrate that the resulting BO procedure shows strong empirical performance on a diverse set of optimization tasks.
Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently
Authors: Xiaohan Chen, Jason Zhang, Zhangyang Wang
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4

Sparse neural networks (NNs) are intensively investigated in literature due to their appeal in saving storage, memory, and computational costs. A recent work (Ramanujan et al., 2020) showed that, different from conventional pruning-and-finetuning pipeline, there exists hidden subnetworks in randomly initialized NNs that have good performance without training the weights. However, such "hidden subnetworks" have mediocre performances and require an expensive edge-popup algorithm to search for them. In this work, we define an extended class of subnetworks in randomly initialized NNs called disguised subnetworks, which are not only "hidden" in the random networks but also "disguised" -- hence can only be "unmasked" with certain transformations on weights. We argue that the unmasking process plays an important role in enlarging the capacity of the subnetworks and thus grants two major benefits: (i) the disguised subnetworks easily outperform the hidden counterparts; (ii) the unmasking process helps to relax the quality requirement on the sparse subnetwork mask so that the expensive edge-popup algorithm can be replaced with more efficient alternatives. On top of the concept of disguised subnetworks, we propose a novel two-stage algorithm that plays a Peek-a-Boo (PaB) game to identify the disguised subnetworks with a combination of two operations: (1) searching efficiently for a subnetwork at random initialization; (2) unmasking the disguise by learning to transform the resulting subnetwork's remaining weights. Furthermore, we show that the unmasking process can be efficiently implemented (a) without referring to any latent weights or scores; and (b) by only leveraging approximated gradients, so that the whole training algorithm is computationally light. Extensive experiments with several large-scale models (ResNet-18, ResNet-50, and WideResNet-28), on CIFAR-10/100 datasets, demonstrate the competency of PaB over edge-popup and other counterparts. Codes will be released upon acceptance.