pwc-archive/papers-with-abstracts huggingface dataset:
Self Sparse Generative Adversarial Networks
Authors: Wenliang Qian, Yang Xu, WangMeng Zuo, Hui Li
Date: 2021-01-26 00:00:00
Tasks: Generative Adversarial Network, Image Generation

Generative Adversarial Networks (GANs) are an unsupervised generative model that learns data distribution through adversarial training. However, recent experiments indicated that GANs are difficult to train due to the requirement of optimization in the high dimensional parameter space and the zero gradient problem. In this work, we propose a Self Sparse Generative Adversarial Network (Self-Sparse GAN) that reduces the parameter space and alleviates the zero gradient problem. In the Self-Sparse GAN, we design a Self-Adaptive Sparse Transform Module (SASTM) comprising the sparsity decomposition and feature-map recombination, which can be applied on multi-channel feature maps to obtain sparse feature maps. The key idea of Self-Sparse GAN is to add the SASTM following every deconvolution layer in the generator, which can adaptively reduce the parameter space by utilizing the sparsity in multi-channel feature maps. We theoretically prove that the SASTM can not only reduce the search space of the convolution kernel weight of the generator but also alleviate the zero gradient problem by maintaining meaningful features in the Batch Normalization layer and driving the weight of deconvolution layers away from being negative. The experimental results show that our method achieves the best FID scores for image generation compared with WGAN-GP on MNIST, Fashion-MNIST, CIFAR-10, STL-10, mini-ImageNet, CELEBA-HQ, and LSUN bedrooms, and the relative decrease of FID is 4.76% ~ 21.84%.
Investigating the significance of adversarial attacks and their relation to interpretability for radar-based human activity recognition systems
Authors: Utku Ozbulak, Baptist Vandersmissen, Azarakhsh Jalalvand, Ivo Couckuyt, Arnout Van Messem, Wesley De Neve
Date: 2021-01-26 00:00:00
Tasks: Activity Recognition, Adversarial Attack, Human Activity Recognition

Given their substantial success in addressing a wide range of computer vision challenges, Convolutional Neural Networks (CNNs) are increasingly being used in smart home applications, with many of these applications relying on the automatic recognition of human activities. In this context, low-power radar devices have recently gained in popularity as recording sensors, given that the usage of these devices allows mitigating a number of privacy concerns, a key issue when making use of conventional video cameras. Another concern that is often cited when designing smart home applications is the resilience of these applications against cyberattacks. It is, for instance, well-known that the combination of images and CNNs is vulnerable against adversarial examples, mischievous data points that force machine learning models to generate wrong classifications during testing time. In this paper, we investigate the vulnerability of radar-based CNNs to adversarial attacks, and where these radar-based CNNs have been designed to recognize human gestures. Through experiments with four unique threat models, we show that radar-based CNNs are susceptible to both white- and black-box adversarial attacks. We also expose the existence of an extreme adversarial attack case, where it is possible to change the prediction made by the radar-based CNNs by only perturbing the padding of the inputs, without touching the frames where the action itself occurs. Moreover, we observe that gradient-based attacks exercise perturbation not randomly, but on important features of the input data. We highlight these important features by making use of Grad-CAM, a popular neural network interpretability method, hereby showing the connection between adversarial perturbation and prediction interpretability.
SkeletonVis: Interactive Visualization for Understanding Adversarial Attacks on Human Action Recognition Models
Authors: Haekyu Park, Zijie J. Wang, Nilaksh Das, Anindya S. Paul, Pruthvi Perumalla, Zhiyan Zhou, Duen Horng Chau
Date: 2021-01-26 00:00:00
Tasks: Action Recognition, Temporal Action Localization

Skeleton-based human action recognition technologies are increasingly used in video based applications, such as home robotics, healthcare on aging population, and surveillance. However, such models are vulnerable to adversarial attacks, raising serious concerns for their use in safety-critical applications. To develop an effective defense against attacks, it is essential to understand how such attacks mislead the pose detection models into making incorrect predictions. We present SkeletonVis, the first interactive system that visualizes how the attacks work on the models to enhance human understanding of attacks.
Probability Trajectory: One New Movement Description for Trajectory Prediction
Authors: Pei Lv, Hui Wei, Tianxin Gu, Yuzhen Zhang, Xiaoheng Jiang, Bing Zhou, Mingliang Xu
Date: 2021-01-26 00:00:00
Tasks: Autonomous Driving, Trajectory Prediction

Trajectory prediction is a fundamental and challenging task for numerous applications, such as autonomous driving and intelligent robots. Currently, most of existing work treat the pedestrian trajectory as a series of fixed two-dimensional coordinates. However, in real scenarios, the trajectory often exhibits randomness, and has its own probability distribution. Inspired by this observed fact, also considering other movement characteristics of pedestrians, we propose one simple and intuitive movement description, probability trajectory, which maps the coordinate points of pedestrian trajectory into two-dimensional Gaussian distribution in images. Based on this unique description, we develop one novel trajectory prediction method, called social probability. The method combines the new probability trajectory and powerful convolution recurrent neural networks together. Both the input and output of our method are probability trajectories, which provide the recurrent neural network with sufficient spatial and random information of moving pedestrians. And the social probability extracts spatio-temporal features directly on the new movement description to generate robust and accurate predicted results. The experiments on public benchmark datasets show the effectiveness of the proposed method.
Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer
Authors: Liang Lin, Yiming Gao, Ke Gong, Meng Wang, Xiaodan Liang
Date: 2021-01-26 00:00:00
Tasks: Graph Representation Learning, Human Parsing, Panoptic Segmentation, Representation Learning, Transfer Learning

Prior highly-tuned image parsing models are usually studied in a certain domain with a specific set of semantic labels and can hardly be adapted into other scenarios (e.g., sharing discrepant label granularity) without extensive re-training. Learning a single universal parsing model by unifying label annotations from different domains or at various levels of granularity is a crucial but rarely addressed topic. This poses many fundamental learning challenges, e.g., discovering underlying semantic structures among different label granularity or mining label correlation across relevant tasks. To address these challenges, we propose a graph reasoning and transfer learning framework, named "Graphonomy", which incorporates human knowledge and label taxonomy into the intermediate graph representation learning beyond local convolutions. In particular, Graphonomy learns the global and structured semantic coherency in multiple domains via semantic-aware graph reasoning and transfer, enforcing the mutual benefits of the parsing across domains (e.g., different datasets or co-related tasks). The Graphonomy includes two iterated modules: Intra-Graph Reasoning and Inter-Graph Transfer modules. The former extracts the semantic graph in each domain to improve the feature representation learning by propagating information with the graph; the latter exploits the dependencies among the graphs from different domains for bidirectional knowledge transfer. We apply Graphonomy to two relevant but different image understanding research topics: human parsing and panoptic segmentation, and show Graphonomy can handle both of them well via a standard pipeline against current state-of-the-art approaches. Moreover, some extra benefit of our framework is demonstrated, e.g., generating the human parsing at various levels of granularity by unifying annotations across different datasets.
Ensembling complex network 'perspectives' for mild cognitive impairment detection with artificial neural networks
Authors: Eufemia Lella, Gennaro Vessio
Date: 2021-01-26 00:00:00

In this paper, we propose a novel method for mild cognitive impairment detection based on jointly exploiting the complex network and the neural network paradigm. In particular, the method is based on ensembling different brain structural "perspectives" with artificial neural networks. On one hand, these perspectives are obtained with complex network measures tailored to describe the altered brain connectivity. In turn, the brain reconstruction is obtained by combining diffusion-weighted imaging (DWI) data to tractography algorithms. On the other hand, artificial neural networks provide a means to learn a mapping from topological properties of the brain to the presence or absence of cognitive decline. The effectiveness of the method is studied on a well-known benchmark data set in order to evaluate if it can provide an automatic tool to support the early disease diagnosis. Also, the effects of balancing issues are investigated to further assess the reliability of the complex network approach to DWI data.
ResLT: Residual Learning for Long-tailed Recognition
Authors: Jiequan Cui, Shu Liu, Zhuotao Tian, Zhisheng Zhong, Jiaya Jia
Date: 2021-01-26 00:00:00
Tasks: Long-tail Learning

Deep learning algorithms face great challenges with long-tailed data distribution which, however, is quite a common case in real-world scenarios. Previous methods tackle the problem from either the aspect of input space (re-sampling classes with different frequencies) or loss space (re-weighting classes with different weights), suffering from heavy over-fitting to tail classes or hard optimization during training. To alleviate these issues, we propose a more fundamental perspective for long-tailed recognition, i.e., from the aspect of parameter space, and aims to preserve specific capacity for classes with low frequencies. From this perspective, the trivial solution utilizes different branches for the head, medium, and tail classes respectively, and then sums their outputs as the final results is not feasible. Instead, we design the effective residual fusion mechanism -- with one main branch optimized to recognize images from all classes, another two residual branches are gradually fused and optimized to enhance images from medium+tail classes and tail classes respectively. Then the branches are aggregated into final results by additive shortcuts. We test our method on several benchmarks, i.e., long-tailed version of CIFAR-10, CIFAR-100, Places, ImageNet, and iNaturalist 2018. Experimental results manifest the effectiveness of our method. Our code is available at https://github.com/jiequancui/ResLT.
AINet+: Advancing Superpixel Segmentation via Cascaded Association Implantation
Authors: Yaxiong Wang, Yunchao Wei, Yujiao Wu, Xueming Qian, Li Zhu, Yi Yang
Date: 2021-01-26 00:00:00
Proceeding: ICCV 2021 10
Tasks: Segmentation

Superpixel segmentation has seen significant progress benefiting from the deep convolutional networks. The typical approach entails initial division of the image into grids, followed by a learning process that assigns each pixel to adjacent grid segments. However, reliance on convolutions with confined receptive fields results in an implicit, rather than explicit, understanding of pixel-grid interactions. This limitation often leads to a deficit of contextual information during the mapping of associations. To counteract this, we introduce the Association Implantation (AI) module, designed to allow networks to explicitly engage with pixel-grid relationships. This module embeds grid features directly into the vicinity of the central pixel and employs convolutional operations on an enlarged window, facilitating an adaptive transfer of knowledge. This approach enables the network to explicitly extract context at the pixel-grid level, which is more aligned with the objectives of superpixel segmentation than mere pixel-wise interactions. By integrating the AI module across various layers, we enable a progressive refinement of pixel-superpixel relationships from coarse to fine. To further enhance the assignment of boundary pixels, we've engineered a boundary-aware loss function. This function aids in the discrimination of boundary-adjacent pixels at the feature level, thereby empowering subsequent modules to precisely identify boundary pixels and enhance overall boundary accuracy. Our method has been rigorously tested on four benchmarks, including BSDS500, NYUv2, ACDC, and ISIC2017, and our model can achieve competitive performance with comparison methods.
Visual explanation of black-box model: Similarity Difference and Uniqueness (SIDU) method
Authors: Satya M. Muddamsetty, Mohammad N. S. Jahromi, Andreea E. Ciontos, Laura M. Fenoy, Thomas B. Moeslund
Date: 2021-01-26 00:00:00
Tasks: Adversarial Attack, Explainable artificial intelligence, Explainable Artificial Intelligence (XAI)

Explainable Artificial Intelligence (XAI) has in recent years become a well-suited framework to generate human understandable explanations of "black-box" models. In this paper, a novel XAI visual explanation algorithm known as the Similarity Difference and Uniqueness (SIDU) method that can effectively localize entire object regions responsible for prediction is presented in full detail. The SIDU algorithm robustness and effectiveness is analyzed through various computational and human subject experiments. In particular, the SIDU algorithm is assessed using three different types of evaluations (Application, Human and Functionally-Grounded) to demonstrate its superior performance. The robustness of SIDU is further studied in the presence of adversarial attack on "black-box" models to better understand its performance. Our code is available at: https://github.com/satyamahesh84/SIDU_XAI_CODE.
Consistent Mesh Colors for Multi-View Reconstructed 3D Scenes
Authors: Mohamed Dahy Elkhouly, Alessio Del Bue, Stuart James
Date: 2021-01-26 00:00:00

We address the issue of creating consistent mesh texture maps captured from scenes without color calibration. We find that the method for aggregation of the multiple views is crucial for creating spatially consistent meshes without the need to explicitly optimize for spatial consistency. We compute a color prior from the cross-correlation of observable view faces and the faces per view to identify an optimal per-face color. We then use this color in a re-weighting ratio for the best-view texture, which is identified by prior mesh texturing work, to create a spatial consistent texture map. Despite our method not explicitly handling spatial consistency, our results show qualitatively more consistent results than other state-of-the-art techniques while being computationally more efficient. We evaluate on prior datasets and additionally Matterport3D showing qualitative improvements.
Towards Universal Physical Attacks On Cascaded Camera-Lidar 3D Object Detection Models
Authors: Mazen Abdelfattah, Kaiwen Yuan, Z. Jane Wang, Rabab Ward
Date: 2021-01-26 00:00:00
Tasks: 3D Object Detection, Adversarial Attack, object-detection, Object Detection, Self-Driving Cars

We propose a universal and physically realizable adversarial attack on a cascaded multi-modal deep learning network (DNN), in the context of self-driving cars. DNNs have achieved high performance in 3D object detection, but they are known to be vulnerable to adversarial attacks. These attacks have been heavily investigated in the RGB image domain and more recently in the point cloud domain, but rarely in both domains simultaneously - a gap to be filled in this paper. We use a single 3D mesh and differentiable rendering to explore how perturbing the mesh's geometry and texture can reduce the robustness of DNNs to adversarial attacks. We attack a prominent cascaded multi-modal DNN, the Frustum-Pointnet model. Using the popular KITTI benchmark, we showed that the proposed universal multi-modal attack was successful in reducing the model's ability to detect a car by nearly 73%. This work can aid in the understanding of what the cascaded RGB-point cloud DNN learns and its vulnerability to adversarial attacks.
Learning Spatial and Spatio-Temporal Pixel Aggregations for Image and Video Denoising
Authors: Xiangyu Xu, Muchen Li, Wenxiu Sun, Ming-Hsuan Yang
Date: 2021-01-26 00:00:00
Tasks: Denoising, Image Denoising, Video Denoising

Existing denoising methods typically restore clear results by aggregating pixels from the noisy input. Instead of relying on hand-crafted aggregation schemes, we propose to explicitly learn this process with deep neural networks. We present a spatial pixel aggregation network and learn the pixel sampling and averaging strategies for image denoising. The proposed model naturally adapts to image structures and can effectively improve the denoised results. Furthermore, we develop a spatio-temporal pixel aggregation network for video denoising to efficiently sample pixels across the spatio-temporal space. Our method is able to solve the misalignment issues caused by large motion in dynamic scenes. In addition, we introduce a new regularization term for effectively training the proposed video denoising model. We present extensive analysis of the proposed method and demonstrate that our model performs favorably against the state-of-the-art image and video denoising approaches on both synthetic and real-world data.
LIGHTS: LIGHT Specularity Dataset for specular detection in Multi-view
Authors: Mohamed Dahy Elkhouly, Theodore Tsesmelis, Alessio Del Bue, Stuart James
Date: 2021-01-26 00:00:00
Tasks: Highlight Detection

Specular highlights are commonplace in images, however, methods for detecting them and in turn removing the phenomenon are particularly challenging. A reason for this, is due to the difficulty of creating a dataset for training or evaluation, as in the real-world we lack the necessary control over the environment. Therefore, we propose a novel physically-based rendered LIGHT Specularity (LIGHTS) Dataset for the evaluation of the specular highlight detection task. Our dataset consists of 18 high quality architectural scenes, where each scene is rendered with multiple views. In total we have 2,603 views with an average of 145 views per scene. Additionally we propose a simple aggregation based method for specular highlight detection that outperforms prior work by 3.6% in two orders of magnitude less time on our dataset.
Lightweight Multi-Branch Network for Person Re-Identification
Authors: Fabian Herzog, Xunbo Ji, Torben Teepe, Stefan Hörmann, Johannes Gilg, Gerhard Rigoll
Date: 2021-01-26 00:00:00
Tasks: Person Re-Identification

Person Re-Identification aims to retrieve person identities from images captured by multiple cameras or the same cameras in different time instances and locations. Because of its importance in many vision applications from surveillance to human-machine interaction, person re-identification methods need to be reliable and fast. While more and more deep architectures are proposed for increasing performance, those methods also increase overall model complexity. This paper proposes a lightweight network that combines global, part-based, and channel features in a unified multi-branch architecture that builds on the resource-efficient OSNet backbone. Using a well-founded combination of training techniques and design choices, our final model achieves state-of-the-art results on CUHK03 labeled, CUHK03 detected, and Market-1501 with 85.1% mAP / 87.2% rank1, 82.4% mAP / 84.9% rank1, and 91.5% mAP / 96.3% rank1, respectively.
CoMo: A novel co-moving 3D camera system
Authors: Andrea Cavagna, Xiao Feng, Stefania Melillo, Leonardo Parisi, Lorena Postiglione, Pablo Villegas
Date: 2021-01-26 00:00:00

Motivated by the theoretical interest in reconstructing long 3D trajectories of individual birds in large flocks, we developed CoMo, a co-moving camera system of two synchronized high speed cameras coupled with rotational stages, which allow us to dynamically follow the motion of a target flock. With the rotation of the cameras we overcome the limitations of standard static systems that restrict the duration of the collected data to the short interval of time in which targets are in the cameras common field of view, but at the same time we change in time the external parameters of the system, which have then to be calibrated frame-by-frame. We address the calibration of the external parameters measuring the position of the cameras and their three angles of yaw, pitch and roll in the system "home" configuration (rotational stage at an angle equal to 0deg and combining this static information with the time dependent rotation due to the stages. We evaluate the robustness and accuracy of the system by comparing reconstructed and measured 3D distances in what we call 3D tests, which show a relative error of the order of 1%. The novelty of the work presented in this paper is not only on the system itself, but also on the approach we use in the tests, which we show to be a very powerful tool in detecting and fixing calibration inaccuracies and that, for this reason, may be relevant for a broad audience.
Dense Semantic Forecasting in Video by Joint Regression of Features and Feature Motion
Authors: Josip Šarić, Sacha Vražić, Siniša Šegvić
Date: 2021-01-26 00:00:00
Tasks: Future prediction, Panoptic Segmentation, regression, Segmentation, Semantic Segmentation

Dense semantic forecasting anticipates future events in video by inferring pixel-level semantics of an unobserved future image. We present a novel approach that is applicable to various single-frame architectures and tasks. Our approach consists of two modules. Feature-to-motion (F2M) module forecasts a dense deformation field that warps past features into their future positions. Feature-to-feature (F2F) module regresses the future features directly and is therefore able to account for emergent scenery. The compound F2MF model decouples the effects of motion from the effects of novelty in a task-agnostic manner. We aim to apply F2MF forecasting to the most subsampled and the most abstract representation of a desired single-frame model. Our design takes advantage of deformable convolutions and spatial correlation coefficients across neighbouring time instants. We perform experiments on three dense prediction tasks: semantic segmentation, instance-level segmentation, and panoptic segmentation. The results reveal state-of-the-art forecasting accuracy across three dense prediction tasks.
Developing emotion recognition for video conference software to support people with autism
Authors: Marc Franzen, Michael Stephan Gresser, Tobias Müller, Prof. Dr. Sebastian Mauser
Date: 2021-01-26 00:00:00
Tasks: Emotion Recognition

We develop an emotion recognition software for the use with a video conference software for autistic individuals which are unable to recognize emotions properly. It can get an image out of the video stream, detect the emotion in it with the help of a neural network and display the prediction to the user. The network is trained on facial landmark features. The software is fully modular to support adaption to different video conference software, programming languages and implementations.