pwc-archive/papers-with-abstracts huggingface dataset:
Causal Scene BERT: Improving object detection by searching for challenging groups
Authors: Cinjon Resnick, Or Litany, Amlan Kar, Karsten Kreis, James Lucas, Kyunghyun Cho, Sanja Fidler
Date: 2021-09-29 00:00:00
Tasks: Autonomous Vehicles, object-detection, Object Detection

Autonomous vehicles (AV) rely on learning-based perception modules parametrized with neural networks for tasks like object detection. These modules frequently have low expected error overall but high error on atypical groups of data due to biases inherent in the training process. Multiple heuristics are employed to identify "failures" in AVs, a typical example being driver interventions. After identification, a human team combs through the associated data to group perception failures that share common causes. More data from these groups is then collected and annotated before retraining the model to fix the issue. In other words, error groups are found and addressed in hindsight as they appear. Our main contribution is a pseudo-automatic method to discover such groups in foresight by performing causal interventions on simulated driving scenes. To keep our interventions on the data manifold, we use masked language models. We verify that the prioritized groups found via intervention are challenging for the object detector and show that retraining with data collected from these groups helps inordinately compared to adding more IID data. We also release software to run interventions in simulated scenes, which we hope will benefit the causality community.
Towards Physical, Imperceptible Adversarial Attacks via Adversarial Programs
Authors: Itai Mesery, Dana Drachsler Cohen
Date: 2021-09-29 00:00:00
Tasks: Program Synthesis

Adversarial examples were originally defined as imperceptible perturbations which cause a deep neural network to misclassify. However, the majority of imperceptible perturbation attacks require to perturb a large number of pixels across the image and are thus hard to execute in the physical world. Existing physical attacks rely on physical objects, such as patches/stickers or 3D-printed objects. Producing adversarial patches is arguably easier than 3D-printing but these attacks incur highly visible perturbations. This raises the question: is it possible to generate adversarial examples with imperceptible patches? In this work, we consider adversarial multi-patch attacks, where the goal is to compute a targeted attack consisting of up to K patches with minimal L2 distortion. Each patch is associated with dimensions, position, and perturbation parameters. We leverage ideas from program synthesis and numerical optimization to search in this large, discrete space and obtain attacks that are competitive with the C&W attack but have at least 3x and up to 10x fewer perturbed pixels. We evaluate our approach on MNIST, Fashion-MNIST, CIFAR-10, and ImageNet and obtain success rate of at least 92% and up to 100% with at most ten patches. For MNIST, Fashion-MNIST, and CIFAR-10, the average L2 distortion is greater than the average L2 distortion of the C&W attack by up to 1.2.
Gradient Explosion and Representation Shrinkage in Infinite Networks
Authors: Adam Klukowski
Date: 2021-09-29 00:00:00

We study deep fully-connected neural networks using the mean field formalism,
and carry out a non-perturbative analysis of signal propagation. As a result, we
demonstrate that increasing the depth leads to gradient explosion or to another
undesirable phenomenon we call representation shrinkage. The appearance of at
least one of these problems is not restricted to a specific initialization scheme or
a choice of activation function, but rather is an inherent property of the fully-
connected architecture itself. Additionally, we show that many popular normal-
ization techniques fail to mitigate these problems. Our method can also be applied
to residual networks to guide the choice of initialization variances.
IDENTIFYING CONCEALED OBJECTS FROM VIDEOS
Authors: Xuelian Cheng, Huan Xiong, Deng-Ping Fan, Yiran Zhong, Mehrtash Harandi, Tom Drummond, ZongYuan Ge
Date: 2021-09-29 00:00:00
Tasks: object-detection, Object Detection

Concealed objects are often hard to identify from still images, as often camouflaged objects exhibit patterns seamless to the background. In this work, we propose a novel video concealed object detection (VCOD) framework, called \textbf{\Ourmodel}, as the concealed state is likely to break when the object moves. The proposed SLT-Net leverages on both short-term dynamics and long-term temporal consistency to detect concealed objects in continuous video frames. Unlike previous methods that often utilize homography or optical flows to explicitly represent motions, we build a dense correlation volume to implicitly capture motions between neighbouring frames. To enforce the temporal consistency within a video sequence, we utilize a spatial-temporal transformer to jointly refine the short-term predictions. Extensive experiments on existing image and VCOD benchmarks demonstrate the architectural effectiveness of our approach. We further collect a large-scale VCOD dataset named MoCA-Mask with pixel-level handcrafted ground-truth masks and construct a comprehensive VCOD benchmark with previous methods. Videos and codes can be found at:  https://anonymous.4open.science/r/long-short-vcod-C0AF/README.md.
Stabilized Self-training with Negative Sampling on Few-labeled Graph Data
Authors: Ziang Zhou, Jieming Shi, Shengzhong Zhang, Zengfeng Huang, Qing Li
Date: 2021-09-29 00:00:00
Tasks: Benchmarking, Node Classification

Graph neural networks (GNNs) are designed for semi-supervised node classification on graphs where only a small subset of nodes have class labels. However, under extreme cases when very few labels are available (e.g., 1 labeled node per class), GNNs suffer from severe result quality degradation. 
Specifically, we observe that existing GNNs suffer from unstable training process on few-labeled graph data, resulting to inferior performance on node classification. Therefore, we propose an effective framework, Stabilized self-training with Negative sampling (SN), which is applicable to existing GNNs to stabilize the training process and enhance the training data, and consequently, boost classification accuracy on graphs with few labeled data. In experiments, we apply our SN framework to two existing GNN base models (GCN and DAGNN) to get SNGCN and SNDAGNN, and evaluate the two methods against 13 existing solutions over 4 benchmarking datasets. Extensive experiments show that the proposed SN framework is highly effective compared with existing solutions, especially under settings with very few labeled data. In particular, on a benchmark dataset Cora with only 1 labeled node per class, while GCN only has 44.6% accuracy, SNGCN achieves 62.5% accuracy, improving GCN by 17.9%; SNDAGNN has accuracy 66.4%, improving that of the base model DAGNN (59.8%) by 6.6%.
Benchmarking Algorithms from Machine Learning for Low-Budget Black-Box Optimization
Authors: Elena Raponi, Nathanaël Carraz Rakotonirina, Jérémy Rapin, Olivier Teytaud, Carola Doerr
Date: 2021-09-29 00:00:00
Tasks: Bayesian Optimization, Benchmarking, BIG-bench Machine Learning, OpenAI Gym

Machine learning has invaded various domains of computer science, including black-box optimization. Recent research is particularly concerned with Bayesian Optimization and/or Monte Carlo Tree Search.
However, the experiments are usually performed on rather small benchmarks and there are visible issues in the experimental setup, such as poor initialization of baselines, overfitting by specifying hyperparameters specifically for each test function, and low statistical significance. In addition, the interface is sometimes very problem-specific and has more impact on the results than the algorithm itself.
We compare several black-box optimization tools from the machine learning world and benchmark them on the classical BBOB benchmark, which is well known in the black-box optimization field, and on Direct Policy Search for OpenAI Gym. In particular, the benchmarks in this work include randomization of the optimum: BBOB considers 15 random instances per test function and dimension, i.e., 24 functions $\times$ 6 dimensionalities $\times$ 15 random instances $=2160$ cases. For OpenAI Gym, we consider tiny and larger neural networks, on a total number of 13 problems $\times$ 8 budgets $\times$ 10 repetitions $=1040$ and 18 problems $\times$ 8 budgets $\times$ 10 repetitions $=1440$ instances, respectively.
Ontology-Driven Semantic Alignment of Artificial Neurons and Visual Concepts
Authors: Riccardo Massidda, Davide Bacciu
Date: 2021-09-29 00:00:00
Tasks: image-classification, Image Classification

Semantic alignment methods attempt to establish a link between human-level concepts and the units of an artificial neural network. Current approaches evaluate the emergence of such meaningful neurons by analyzing the effect of semantically annotated inputs on their activations. In doing so, they often understate two aspects that characterize neural representations and semantic concepts, namely the distributed nature of the former and the existence of semantic relationships binding the latter. In this work, we explicitly tackle this interrelatedness, both at a neural and a conceptual level, by providing a novel semantic alignment framework that builds on aligning a structured ontology with the distributed neural representations. The ontology introduces semantic relations between concepts, enabling the clustering of topologically related units into semantically rich and meaningful neural circuits. Our empirical analysis on notable convolutional models for image classification discusses the emergence of such neural circuits. It also validates their meaningfulness by studying how the selected units are pivotal for the accuracy of classes that are semantically related to the aligned concepts. We also contribute by releasing the code implementing our alignment framework.
Perturbation Deterioration: The Other Side of Catastrophic Overfitting
Authors: Zichao Li, Liyuan Liu, chengyu dong, Jingbo Shang
Date: 2021-09-29 00:00:00

Our goal is to understand why the robustness accuracy would abruptly drop to zero, after conducting FGSM-style adversarial training for too long.  While this phenomenon is commonly explained as overfitting, we observe that it is a twin process: not only does the model catastrophic overfits to one type of perturbation, but also the perturbation deteriorates into random noise. For example, at the same epoch when the FGSM-trained model catastrophically overfits, its generated perturbations deteriorate into random noise. Intuitively, once the generated perturbations become weak and inadequate, models would be misguided to overfit those weak attacks and fail to defend strong ones. In the light of our analyses, we propose APART, an adaptive adversarial training method, which parameterizes perturbation generation and progressively strengthens them.  In our experiments, APART successfully prevents perturbation deterioration and catastrophic overfitting.  Also, APART significantly improves the model robustness while maintaining the same efficiency as FGSM-style methods, e.g., on the CIFAR-10 dataset, APART achieves 53.89%accuracy under the PGD-20 attack and 49.05% accuracy under the AutoAttack.
Unsupervised Visual Program Induction with Function Modularization
Authors: Xuguang Duan, Xin Wang, Ziwei Zhang, Wenwu Zhu
Date: 2021-09-29 00:00:00
Tasks: Program induction

Program induction serves as one way to analog the ability of human thinking. However, existing methods could only tackle the task under simple scenarios (Fig~\ref{fig:task_examples}(a),(b)). When it comes to complex scenes, e.g., the visual scenes, current program induction methods fail due to the huge program action space. In this paper, to the best of our knowledge, we are the first to tackle this problem. We propose a novel task named {\it unsupervised visual program induction} in complex visual scenes that require complex primitive functions. Solving this task faces two challenges: i) modeling complex primitive functions for complex visual scenes is very difficult, and ii) employing complex functions in the unsupervised program induction suffers from a huge and heterogeneous program action space. To tackle these challenges, we propose the Self-Exploratory-Modularized-Function (SEMF) model, which can jointly model individual function selection and its parameters through a unified modular block. Moreover, a Monto-Carlo-Tree-Search (MCTS) based Self-Exploratory algorithm is proposed to explore program space with modularized function as prior. The exploratory results, in turn, guide the training of these modularized functions. Our extensive experiments demonstrate that the proposed SEFM model outperforms all the existing baselines in model performance, training efficiency, and model generalization ability.
EXPLAINABLE AI-BASED DYNAMIC FILTER PRUNING OF CONVOLUTIONAL NEURAL NETWORKS
Authors: Muhammad Sabih, Frank Hannig, Jürgen Teich
Date: 2021-09-29 00:00:00
Tasks: CPU, GPU, image-classification, Image Classification

Filter pruning is one of the most effective ways to accelerate Convolutional Neural Networks (CNNs). Most of the existing works are focused on the static pruning of CNN filters. In dynamic pruning of CNN filters, existing works are based on
the idea of switching between different branches of a CNN or exiting early based on the difficulty of a sample. These approaches can reduce the average latency of inference, but they cannot reduce the longest-path latency of inference. In contrast, we present a novel approach of dynamic filter pruning that utilizes explainable AI along with early coarse prediction in the intermediate layers of a CNN. This coarse prediction is performed using a simple branch that is trained to perform top-k classification. The branch either predicts the output class with high confidence, in which case, the rest of the computations are left out. Alternatively, the branch predicts the output class to be within a subset of possible output classes. After this coarse prediction, only those filters that are important for this subset of classes are utilized for further computations. The importances of filters for each output class are obtained using explainable AI. Using this architecture of dynamic pruning, we not only reduce the average latency of inference, but we can also reduce the longest-path latency of inference. Our proposed architecture for dynamic pruning can be deployed on different hardware platforms. We evaluate our approach using commonly used image classification models and datasets on CPU and GPU platforms and demonstrate speedup without significant overhead.
INFERNO: Inferring Object-Centric 3D Scene Representations without Supervision
Authors: Lluis Castrejon, Nicolas Ballas, Aaron Courville
Date: 2021-09-29 00:00:00
Tasks: Object, Video Object Tracking, Visual Reasoning

We propose INFERNO, a method to infer object-centric representations of visual scenes without relying on annotations. Our method learns to decompose a scene into multiple objects, each object having a structured representation that disentangles its shape, appearance and 3D pose. To impose this structure we rely on recent advances in neural 3D rendering. Each object representation defines a localized neural radiance field that is used to generate 2D views of the scene through a differentiable rendering process. Our model is subsequently trained by minimizing a reconstruction loss between inputs and corresponding rendered scenes. We empirically show that INFERNO discovers objects in a scene without supervision. We also validate the interpretability of the learned representations by manipulating inferred scenes and showing the corresponding effect in the rendered output. Finally, we demonstrate the usefulness of our 3D object representations in a visual reasoning task using the CATER dataset.
FoxInst: A Frustratingly Simple Baseline for Weakly Few-shot Instance Segmentation
Authors: Dongmin Choi, Moon Ye-Bin, Junsik Kim, Tae-Hyun Oh
Date: 2021-09-29 00:00:00
Tasks: Few-shot Instance Segmentation, Instance Segmentation, Semantic Segmentation

We propose the first weakly-supervised few-shot instance segmentation task and a frustratingly simple but strong baseline model, FoxInst. Our work is distinguished from other approaches in that our method is trained with weak annotations, i.e., class and box annotations, during all phases, which leads to further data efficiency and practicality. Considering the challenging regime of our problem, we design the network to be an anchor-free architecture to avoid anchor box restriction, and train the network in a simple and stable way that first trains the whole network on the base classes, and then only fine-tunes the heads partially with few novel class data. To establish the foundation as a strong baseline, we carefully design evaluation setups by correcting the existing problems in the evaluation metric and test set, so that the effects of each component are well revealed. We show that FoxInst achieves comparable or even higher performance with the prior fully-supervised FSIS networks on COCO and PASCAL VOC datasets. We will release the code if accepted for reproduction.
Adversarial Style Transfer for Robust Policy Optimization in Reinforcement Learning
Authors: Md Masudur Rahman, Yexiang Xue
Date: 2021-09-29 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Style Transfer

This paper proposes an algorithm that aims to improve generalization for reinforcement learning agents by removing overfitting to confounding features. Our approach consists of a max-min game theoretic objective. A generator transfers the style of observation during reinforcement learning. An additional goal of the generator is to perturb the observation, which maximizes the agent's probability of taking a different action. In contrast, a policy network updates its parameters to minimize the effect of such perturbations, thus staying robust while maximizing the expected future reward. Based on this setup, we propose a practical deep reinforcement learning algorithm, Adversarial Robust Policy Optimization (ARPO), to find an optimal policy that generalizes to unseen environments. We evaluate our approach on visually enriched and diverse Procgen benchmarks. Empirically, we observed that our agent ARPO performs better in generalization and sample efficiency than a few state-of-the-art algorithms.
Towards fast and effective single-step adversarial training
Authors: Pau de Jorge, Adel Bibi, Riccardo Volpi, Amartya Sanyal, Philip Torr, Grégory Rogez, Puneet K. Dokania
Date: 2021-09-29 00:00:00

Recently, Wong et al. (2020) showed adversarial training with single-step FGSM leads  to  a  characteristic  failure  mode  named catastrophic  overfitting (CO),  in which  a  model  becomes  suddenly  vulnerable  to  multi-step  attacks.   Moreover, they showed adding a random perturbation prior to FGSM (RS-FGSM) seemed to be sufficient to prevent CO. However, Andriushchenko & Flammarion (2020) observed that RS-FGSM still leads to CO for larger perturbations and argue that the  only  contribution  of  the  random  step  is  to  reduce  the  magnitude  of  the  attacks.  They suggest a regularizer (GradAlign) that avoids CO but is significantly more expensive than RS-FGSM. In this work,  we methodically revisit the role of noise and clipping in single-step adversarial training.  Contrary to previous intuitions, we find that not clipping the perturbation around the clean sample and using a stronger noise is highly effective in avoiding CO for large perturbation radii,  despite leading to an increase in the magnitude of the attacks.   Based on these observations, we propose a method called Noise-FGSM (N-FGSM), which attacks noise-augmented samples directly using a single-step. Empirical analyses on a large suite of experiments show that N-FGSM is able to match or surpass the performance of GradAlign while achieving a 3x speed-up.
Informative Robust Causal Representation for Generalizable Deep Learning
Authors: Mengyue Yang, Furui Liu, Xu Chen, Zhitang Chen, Jianye Hao, Jun Wang
Date: 2021-09-29 00:00:00
Tasks: counterfactual, Deep Learning, image-classification, Image Classification, Recommendation Systems, Representation Learning

In many real-world scenarios, such as image classification and recommender systems, it is evidence that representation learning can improve model's performance over multiple downstream tasks. Existing learning approaches rely on establishing the correlation (or its proxy) between features and the downstream task (labels), which typically  results in a representation containing cause, effect and spurious correlated variables of the label. Its generalizability may deteriorate because of the unstability of the non-causal parts. In this paper, we propose to learn causal representation from observational data by regularizing the learning procedure with mutual information measures according to our hypothetical causal graph. The optimization involves a counterfactual loss, based on which we deduce a theoretical guarantee that the causality-inspired learning is with reduced sample complexity and better generalization ability. Extensive experiments show that the models trained on causal representations learned by our approach is  robust under adversarial attacks and distribution shift.
A Class of Short-term Recurrence Anderson Mixing Methods and Their Applications
Authors: Fuchao Wei, Chenglong Bao, Yang Liu
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: image-classification, Image Classification, Stochastic Optimization

Anderson mixing (AM) is a powerful acceleration method for fixed-point iterations, but its computation requires storing many historical iterations. The extra memory footprint can be prohibitive when solving high-dimensional problems in a resource-limited machine. To reduce the memory overhead, we propose  a novel class of short-term recurrence AM methods (ST-AM). The ST-AM methods only store two previous iterations with cheap corrections. We prove that the basic version of ST-AM is equivalent to the full-memory AM in strongly convex quadratic optimization,  and with minor changes it has local linear convergence for solving general nonlinear fixed-point problems. These properties guarantee the theoretical advantages of ST-AM over classical gradient-based methods. We further propose the regularized ST-AM and analyze its convergence properties for nonconvex (stochastic) optimization. Finally, we apply ST-AM to several applications including training neural networks in image classification and language tasks. Experimental results show that ST-AM is competitive with the long-memory AM and outperforms many existing optimizers, including SGD and ADAM.