pwc-archive/papers-with-abstracts huggingface dataset:
Actor-Critic Policy Optimization in a Large-Scale Imperfect-Information Game
Authors: Haobo Fu, Weiming Liu, Shuang Wu, Yijia Wang, Tao Yang, Kai Li, Junliang Xing, Bin Li, Bo Ma, Qiang Fu, Yang Wei
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: counterfactual, Deep Reinforcement Learning, Policy Gradient Methods

The deep policy gradient method has demonstrated promising results in many large-scale games, where the agent learns purely from its own experience. Yet, policy gradient methods suffer convergence problems to a Nash Equilibrium (NE) in multi-agent situations. Counterfactual regret minimization has a convergence guarantee to a NE in two-player zero-sum games, but it usually needs domain-specific abstraction techniques and model-based traversing to deal with large-scale games.  To inherit merits from both methods, we extend the actor-critic algorithm framework in deep reinforcement learning to solve a large-scale two-player zero-sum imperfect-information game, 1v1 Mahjong, whose information set size and game length are much larger than Poker. In particular, we modify the policy optimization objective from originally maximizing the discounted returns to minimizing a type of weighted cumulative counterfactual regrets. This modification is achieved by approximating the regrets via a deep neural network and minimizing the regrets via generating self-play strategies using Hedge. We name the proposed algorithm Actor-Critic Hedge (ACH) and derive its theoretical connection to CFR. We prove the convergence of ACH to a NE under certain conditions.  Experimental results on the proposed 1v1 Mahjong benchmark and benchmarks from OpenSpiel demonstrate that ACH outperforms related state-of-the-art methods. Also, the bot obtained by ACH defeats a human champion in 1v1 Mahjong.
Interpretable Semantic Role Relation Table for Supporting Facts Recognition of Reading Comprehension
Authors: YanQing Bai, Zhichang Zhang, HaoYuan Chen, Xiaohui Qin, Yanglong Qiu
Date: 2021-09-29 00:00:00
Tasks: Machine Reading Comprehension, Reading Comprehension, Relation, Semantic Role Labeling, Sentence

The current Machine Reading Comprehension (MRC) model has poor interpretability. 
Interpretable semantic features can enhancethe interpretability of the model. 
Semantic role labeling (SRL) captures predicate-argument relations, such as "who did what to whom," which
are critical to comprehension and interpretation. 
To enhance the interpretability of the model, we propose the semantic role relation table,
which represents the semantic relation of the sentence itself and the semantic relations among sentences. 
We use the name of entities to integrate into the semantic role relation table to establish the semantic relation between sentences. 
This paper makes the first attempt to utilize contextual semantic's explicit relation to the recognition supporting fact of reading 
comprehension. 
We have established nine semantic relationtables between target sentence, question, and article. 
Then we take each semantic relationship table's overall semantic role relevance and each
semantic role relevance as important judgment information. 
Detailed experiments on HotpotQA, a challenging multi-hop MRC data set, our method achieves better performance. 
With few training data sets, the model performance is still stable.
Learning From Unpaired Data: A Variational Bayes Approach
Authors: Dihan Zheng, Xiaowen Zhang, Kaisheng Ma, Chenglong Bao
Date: 2021-09-29 00:00:00
Tasks: Denoising, Image Denoising, Super-Resolution, Variational Inference

Collecting the paired training data is a difficult task in practice, but the unpaired samples broadly exist. Thus, current approaches aim at generating synthesized training data from the unpaired samples by exploring the relationship between the corrupted and clean data. In this work, we propose LUD-VAE, a method to learn the joint probability density function from the data sampled from marginal distributions. Our method is based on the variational inference framework and maximizes the evidence lower bound (ELBO), the lower bound of the joint probability density function. Furthermore, we show that the ELBO is computable without paired samples under the inference invariant assumption. This property provides the mathematical rationale of our approach in the unpaired setting. Finally, we apply our method to the real-world image denoising and super-resolution tasks and train the models using the synthetic data generated by the LUD-VAE. Experimental results on four datasets validate the advantages of our method over other learnable approaches.
Adaptive Label Smoothing with Self-Knowledge
Authors: Dongkyu Lee, Ka Chun Cheung, Nevin Zhang
Date: 2021-09-29 00:00:00
Tasks: Knowledge Distillation, Machine Translation

Overconfidence has been shown to impair generalization and calibration of a neural network. Previous studies remedy this issue by adding a regularization term to a loss function, preventing a model from making a peaked distribution. Label smoothing smoothes target labels with a predefined prior label distribution; as a result, a model is learned to maximize the likelihood of predicting the soft label. Nonetheless, the amount of smoothing is the same in all samples and remains fixed in training. In other words, label smoothing does not reflect the change in probability distribution mapped by a model over the course of training. To address this issue, we propose a regularization scheme that brings dynamic nature into the smoothing parameter by taking model probability distribution into account, thereby varying the parameter per instance. A model in training self-regulates the extent of smoothing on the fly during forward propagation. Furthermore, inspired by recent work in bridging label smoothing and knowledge distillation, our work utilizes self-knowledge as a prior label distribution in softening target labels, and presents theoretical support for the regularization effect by knowledge distillation. Our regularizer is validated comprehensively on various datasets in machine translation and outperforms strong baselines not only in model performance but also in model calibration by a large margin.
CDPS: Constrained DTW-Preserving Shapelets
Authors: Hussein El Amouri, Thomas Lampert, Pierre Gançarski, Clement Mallet
Date: 2021-09-29 00:00:00
Tasks: Clustering, Constrained Clustering, Dynamic Time Warping, GPU, Representation Learning, Time Series, Time Series Analysis

The analysis of time series for clustering and classification is becoming ever more popular because of the increasingly ubiquitous nature of IoT, satellite constellations, and handheld and smart-wearable devices, etc. Euclidean distance is unsuitable because of potential phase shift, differences in sample duration, and compression and dilation of characteristic signals. As such, several similarity measures specific to time-series have been proposed, Dynamic Time Warping (DTW) being the most popular. Nevertheless, DTW does not respect the axioms of a metric and therefore DTW-preserving shapelets have been developed to regain these properties. This unsupervised approach to representation learning models DTW properties through the shapelet transform. This article proposes constrained DTW-preserving shapelets (CDPS), in which a limited amount of user knowledge is available in the form of must link and cannot link constraints, to guide the representation such that it better captures the user’s interpretation of the data rather than the algorithm’s bias. Subsequently, any unconstrained algorithm can be applied, e.g. K-means clustering, k-NN classification, etc, to obtain a result that fulfills the constraints (without explicit knowledge of them). Furthermore, this representation is generalisable to out-of-sample data, overcoming the limitations of standard transductive constrained-clustering algorithms. The proposed algorithm is studied on multiple time-series datasets, and its advantages over classical constrained clustering algorithms and unsupervised DTW-preserving shapelets are demonstrated. An open-source implementation based on PyTorch is available to take full advantage of GPU acceleration
Efficient Bi-level Optimization for Non-smooth Optimization
Authors: Wanli Shi, Heng Huang, Bin Gu
Date: 2021-09-29 00:00:00
Tasks: Open-Ended Question Answering

Bi-level optimization plays a key role in a lot of machine learning applications.  However, existing state-of-the-art bi-level optimization methods are limited to smooth or some specific non-smooth lower-level problems.  Even worse, most of them depend on approximating hypergradients to update upper-level variable which is the inherent reason for non-efficiency. Currently, achieving a generalized and efficient optimization algorithm for bi-level problems with a non-smooth, even non-Lipschitz continuous lower-level objective is still an open question to the best of our knowledge. To address these challenging problems, in this paper, we propose a new bi-level optimization algorithm based on the smoothing and penalty techniques. Specifically, we first produce a sequence of smoothed lower-level objectives with an exponential decay smoothing parameter for the non-smooth lower-level problem. Then, we transform the smoothed bi-level optimization to an unconstrained penalty problem by replacing the smoothed sub-problem with its first-order necessary conditions. Finally, we update the upper and lower-level variables alternately with doubly stochastic gradients of the unconstrained penalty problem. Importantly, we provide the theoretical analysis to show that our method can converge to a stationary point of original non-smooth bi-level problem if the lower-level problem is convex, and we give the necessary condition of the original problem if the lower-level problem is nonconvex. We compare our method with existing state-of-the-art bi-level optimization methods in three tasks, and all the experimental results demonstrate that our method is superior to the others in terms of accuracy and efficiency.
Adversarial twin neural networks: maximizing physics recovery for physical system
Authors: Haoran Li, Erik Blasch, Jingyi Yuan, Yang Weng
Date: 2021-09-29 00:00:00

The exact modeling of modern physical systems is challenging due to the expanding system territory and insufficient sensors. To tackle this problem, existing methods utilize sparse regression to find physical parameters or add another virtual learning model like a Neural Network (NN) to universally approximate the unobserved physical quantities. However, the two models can't perfectly play their own roles in joint learning without proper restrictions. Thus, we propose (1) sparsity regularization for the physical model and (2) physical superiority over the virtual model. They together define output boundaries for the physical and virtual models. Further, even the two models output properly, the joint model still can't guarantee learning maximal physical knowledge. For example, if the data of an observed node can linearly represent those of an unobserved node, these two nodes can be aggregated. Therefore, we propose (3) to seek the dissimilarity of physical and virtual outputs to obtain maximal physics. To achieve goals (1)-(3), we design a twin structure of the Physical Neural Network (PNN) and Virtual Neural Network (VNN), where sparse regularization and skip-connections are utilized to guarantee (1) and (2). Then, we propose an adversarial learning scheme to maximize output dissimilarity, achieving (3). We denote the model as the Adversarial Twin Neural Network (ATN).  Finally, we conduct extensive experiments over various systems to demonstrate the best performance of ATN over other state-of-the-art methods.
Use of small auxiliary networks and scarce data to improve the adversarial robustness of deep learning models
Authors: Davide Coppola, Hwee Kuan Lee, Cuntai Guan
Date: 2021-09-29 00:00:00
Tasks: Adversarial Robustness, image-classification, Image Classification

Deep Learning models for image classification are known to be vulnerable to adversarial examples. 
Adversarial training is one of the most effective ways to provide defense against such threats, however it is a cumbersome process which requires many data points and long computation times. 
In a setting where only small amounts of data are available for this process, adversarial training may negatively impact the classification performance on clean images by overfitting on the small amount of data.
This would be undesirable, especially when a large pre-trained model with satisfactory performance on clean data is already available.
We propose a new strategy to make a previously-trained model more robust against adversarial attacks, using scarce data and without degrading its performance on clean samples.
The proposed strategy consists in freezing the parameters of the originally trained base model and adding small auxiliary networks along the architecture, which process the features to reduce the effect of any adversarial perturbation.
This method can be used to defend a model against any arbitrary attack.
A practical advantage of using auxiliary networks is that no modifications on the originally trained base model is required. 
Therefore, it can serve as a patch or add on to fix large and expensive existing deep learning models with little additional resources.
Experiments on the CIFAR10 dataset showed that using only $10\%$ of the full training set, the proposed method was able to adequately defend the model against the AutoPGD attack while maintaining a classification accuracy on clean images outperforming the model with adversarial training by $7\%$. Indeed, the proposed method still performs reasonably well compared to adversarial training using $1\%$ of the full training set.
AlignMix: Improving representations by interpolating aligned features
Authors: Shashanka Venkataramanan, Ewa Kijak, Laurent Amsaleg, Yannis Avrithis
Date: 2021-09-29 00:00:00
Tasks: Data Augmentation, Representation Learning

Mixup is a powerful data augmentation method that interpolates between two or more examples in the input or feature space and between the corresponding target labels. However, how to best interpolate images is not well defined. Recent mixup methods overlay or cut-and-paste two or more objects into one image, which needs care in selecting regions. Mixup has also been connected to autoencoders, because often autoencoders generate an image that continuously deforms into another. However, such images are typically of low quality.

In this work, we revisit mixup from the deformation perspective and introduce AlignMix, where we geometrically align two images in the feature space. The correspondences allow us to interpolate between two sets of features, while keeping the locations of one set. Interestingly, this retains mostly the geometry or pose of one image and the appearance or texture of the other. We also show that an autoencoder can still improve representation learning under mixup, without the classifier ever seeing decoded images. AlignMix outperforms state-of-the-art mixup methods on five different benchmarks.
Sequential Communication in Multi-Agent Reinforcement Learning
Authors: Ziluo Ding, Weixin Hong, Liwen Zhu, Tiejun Huang, Zongqing Lu
Date: 2021-09-29 00:00:00
Tasks: Decision Making, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Coordination is one of the essential problems in multi-agent reinforcement learning. Communication provides an alternative for agents to obtain information about others so that better coordinated behavior can be learned. Some existing work communicates predicted future trajectory with others, hoping to get clues about what others would do for better coordination. However, circular dependencies can inevitably occur when agents are treated equally so that it is impossible to coordinate decision-making. In this paper, we propose a novel communication scheme Sequential Communication (SeqComm). In more detail, we treat agents unequally (the upper-level agents make decisions prior to the lower-level) and have two communication phases. In the negotiation phase, agents share observations with others and obtain their intention by modeling the environment dynamics. Agents determine the priority of decision-making by comparing the value of intention. In the launching phase, the upper-level agents take the lead in making decisions and share their actions with the lower-level agents. Empirically, we show that SeqComm improves the performance in a variety of multi-agent cooperative scenarios, comparing to existing methods.
Isotropic Contextual Representations through Variational Regularization
Authors: Cornelia Ferner, Stefan Wegenkittl
Date: 2021-09-29 00:00:00
Tasks: Decoder, Sentence

Contextual language representations achieve state-of-the-art performance across various natural language processing tasks. However, these representations have been shown to suffer from the degeneration problem, i.e. they occupy a narrow cone in the latent space. This problem can be addressed by enforcing isotropy in the latent space. In analogy to variational autoencoders, we suggest applying a token-level variational loss to a Transformer architecture and introduce the prior distribution's standard deviation as model parameter to optimize isotropy. The encoder-decoder architecture allows for learning interpretable embeddings that can be decoded into text again. Extracted features at sentence-level achieve competitive results on benchmark classification tasks.
Adversarially Robust Conformal Prediction
Authors: Asaf Gendler, Tsui-Wei Weng, Luca Daniel, Yaniv Romano
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Adversarial Attack, Conformal Prediction, Prediction, valid

Conformal prediction is a model-agnostic tool for constructing prediction sets that are valid under the common i.i.d. assumption, which has been applied to quantify the prediction uncertainty of deep net classifiers. In this paper, we generalize this framework to the case where adversaries exist during inference time, under which the i.i.d. assumption is grossly violated. By combining conformal prediction with randomized smoothing, our proposed method forms a prediction set with finite-sample coverage guarantee that holds for any data distribution with $\ell_2$-norm bounded adversarial noise, generated by any adversarial attack algorithm. The core idea is to bound the Lipschitz constant of the non-conformity score by smoothing it with Gaussian noise and leverage this knowledge to account for the effect of the unknown adversarial perturbation. We demonstrate the necessity of our method in the adversarial setting and the validity of our theoretical guarantee on three widely used benchmark data sets: CIFAR10, CIFAR100, and ImageNet.
Learning mixture of neural temporal point processes for event sequence clustering
Authors: Yunhao Zhang, Junchi Yan, Zhenyu Ren, Jian Yin
Date: 2021-09-29 00:00:00
Tasks: Clustering, Point Processes

Event sequence clustering applies to many scenarios e.g. e-Commerce and electronic health. Traditional clustering models fail to characterize complex real-world processes due to the strong parametric assumption. While Neural Temporal Point Processes (NTPPs) mainly focus on modeling similar sequences instead of clustering. To fill the gap, we propose Mixture of Neural Temporal Point Processes (NTPP-MIX), a  general framework that can utilize many existing NTPPs for event sequence clustering. In NTPP-MIX, the prior distribution of coefficients for cluster assignment is modeled by a Dirichlet distribution. When the assignment is given, the conditional probability of a sequence is modeled by the mixture of series of NTPPs. We combine variational EM algorithm and Stochastic Gradient Descent (SGD) to efficiently train the framework. Moreover, to further improve its capability, we propose a fully data-driven NTPP based on the attention mechanism named Fully Attentive Temporal Point Process (FATPP). Experiments on both synthetic and real-world datasets show the effectiveness of NTPP-MIX against state-of-the-arts, especially when using FATPP as a basic NTPP module.
I-PGD-AT: Efficient Adversarial Training via Imitating Iterative PGD Attack
Authors: Xiaosen Wang, Bhavya Kailkhura, Krishnaram Kenthapadi, Bo Li
Date: 2021-09-29 00:00:00

Adversarial training has been widely used in various machine learning paradigms to improve the robustness; while it would increase the training cost due to the perturbation optimization process. To improve the efficiency, recent studies leverage Fast Gradient Sign Method with Random Start (FGSM-RS) for adversarial training. However, such methods would lead to relatively low robustness and catastrophic overfitting, which means the robustness against iterative attacks (e.g. Projected Gradient Descent (PGD)) would suddenly drop to 0%. Different approaches have been proposed to address this problem, while later studies show that catastrophic overfitting still remains. In this paper, motivated by the fact that expensive iterative adversarial training methods achieve high robustness without catastrophic overfitting, we aim to ask: Can we perform iterative adversarial training in an efficient way? To this end, we first analyze the difference of perturbation generated by FGSM-RS and PGD and find that PGD tends to craft diverse discrete values instead of $\pm 1$ in FGSM-RS. Based on this observation, we propose an efficient single-step adversarial training method I-PGD-AT by adopting I-PGD attack for training, in which I-PGD imitates PGD virtually. Unlike FGSM that crafts the perturbation directly using the sign of gradient, I-PGD imitates the perturbation of PGD based on the magnitude of gradient. Extensive empirical evaluations on CIFAR-10 and Tiny ImageNet demonstrate that our I-PGD-AT can improve the robustness compared with the baselines and significantly delay catastrophic overfitting. Moreover, we explore and discuss the factors that affect catastrophic overfitting. Finally, to demonstrate the generality of I-PGD-AT, we integrate it into PGD adversarial training and show that it can even further improve the robustness.
Defending Graph Neural Networks via Tensor-Based Robust Graph Aggregation
Authors: Jianfu Zhang, Yan Hong, Dawei Cheng, Liqing Zhang, Qibin Zhao
Date: 2021-09-29 00:00:00

Graph Neural Networks (GNNs) have achieved outstanding success in a wide variety of domains and applications. However, they are still vulnerable to unnoticeable perturbations of graphs specially designed by attackers, causing significant performance drops. Developing algorithms to defend GNNs with robust graphs vaccinating from adversarial attacks still remains a challenging issue. Existing methods treat every edges individually and regularize them by specific robust properties, which ignores the structural relationships among edges and correlations among different properties. In this paper, we propose a tensor-based framework for GNNs to learn robust graphs from adversarial graphs by aggregating predefined robust graphs to enhance the robustness of GNNs via tensor approximation. All the predefined robust graphs are linearly compressed into and recovered from a low-rank space, which aggregates the robust graphs and the structural information in a balanced manner. Extensive experiments on real-world graph datasets show that the proposed framework effectively mitigates the adverse effects of adversarial attacks and outperforms state-of-the-art defense methods.