pwc-archive/papers-with-abstracts huggingface dataset:
Generative Dynamic Patch Attack
Authors: Xiang Li, Shihao Ji
Date: 2021-11-08 00:00:00

Adversarial patch attack is a family of attack algorithms that perturb a part of image to fool a deep neural network model. Existing patch attacks mostly consider injecting adversarial patches at input-agnostic locations: either a predefined location or a random location. This attack setup may be sufficient for attack but has considerable limitations when using it for adversarial training. Thus, robust models trained with existing patch attacks cannot effectively defend other adversarial attacks. In this paper, we first propose an end-to-end patch attack algorithm, Generative Dynamic Patch Attack (GDPA), which generates both patch pattern and patch location adversarially for each input image. We show that GDPA is a generic attack framework that can produce dynamic/static and visible/invisible patches with a few configuration changes. Secondly, GDPA can be readily integrated for adversarial training to improve model robustness to various adversarial attacks. Extensive experiments on VGGFace, Traffic Sign and ImageNet show that GDPA achieves higher attack success rates than state-of-the-art patch attacks, while adversarially trained model with GDPA demonstrates superior robustness to adversarial patch attacks than competing methods. Our source code can be found at https://github.com/lxuniverse/gdpa.
Adaptive area-preserving parameterization of open and closed anatomical surfaces
Authors: Gary P. T. Choi, Amita Giri, Lalan Kumar
Date: 2021-11-08 00:00:00
Tasks: Object

The parameterization of open and closed anatomical surfaces is of fundamental importance in many biomedical applications. Spherical harmonics, a set of basis functions defined on the unit sphere, are widely used for anatomical shape description. However, establishing a one-to-one correspondence between the object surface and the entire unit sphere may induce a large geometric distortion in case the shape of the surface is too different from a perfect sphere. In this work, we propose adaptive area-preserving parameterization methods for simply-connected open and closed surfaces with the target of the parameterization being a spherical cap. Our methods optimize the shape of the parameter domain along with the mapping from the object surface to the parameter domain. The object surface will be globally mapped to an optimal spherical cap region of the unit sphere in an area-preserving manner while also exhibiting low conformal distortion. We further develop a set of spherical harmonics-like basis functions defined over the adaptive spherical cap domain, which we call the adaptive harmonics. Experimental results show that the proposed parameterization methods outperform the existing methods for both open and closed anatomical surfaces in terms of area and angle distortion. Surface description of the object surfaces can be effectively achieved using a novel combination of the adaptive parameterization and the adaptive harmonics. Our work provides a novel way of mapping anatomical surfaces with improved accuracy and greater flexibility. More broadly, the idea of using an adaptive parameter domain allows easy handling of a wide range of biomedical shapes.
Cross-Modal Object Tracking: Modality-Aware Representations and A Unified Benchmark
Authors: Chenglong Li, Tianhao Zhu, Lei Liu, Xiaonan Si, Zilin Fan, Sulan Zhai
Date: 2021-11-08 00:00:00
Tasks: Object Tracking, Visual Tracking

In many visual systems, visual tracking often bases on RGB image sequences, in which some targets are invalid in low-light conditions, and tracking performance is thus affected significantly. Introducing other modalities such as depth and infrared data is an effective way to handle imaging limitations of individual sources, but multi-modal imaging platforms usually require elaborate designs and cannot be applied in many real-world applications at present. Near-infrared (NIR) imaging becomes an essential part of many surveillance cameras, whose imaging is switchable between RGB and NIR based on the light intensity. These two modalities are heterogeneous with very different visual properties and thus bring big challenges for visual tracking. However, existing works have not studied this challenging problem. In this work, we address the cross-modal object tracking problem and contribute a new video dataset, including 654 cross-modal image sequences with over 481K frames in total, and the average video length is more than 735 frames. To promote the research and development of cross-modal object tracking, we propose a new algorithm, which learns the modality-aware target representation to mitigate the appearance gap between RGB and NIR modalities in the tracking process. It is plug-and-play and could thus be flexibly embedded into different tracking frameworks. Extensive experiments on the dataset are conducted, and we demonstrate the effectiveness of the proposed algorithm in two representative tracking frameworks against 17 state-of-the-art tracking methods. We will release the dataset for free academic usage, dataset download link and code will be released soon.
Template NeRF: Towards Modeling Dense Shape Correspondences from Category-Specific Object Images
Authors: Jianfei Guo, Zhiyuan Yang, Xi Lin, Qingfu Zhang
Date: 2021-11-08 00:00:00
Tasks: 3D-Aware Image Synthesis, Image Generation, Keypoint Detection, NeRF

We present neural radiance fields (NeRF) with templates, dubbed Template-NeRF, for modeling appearance and geometry and generating dense shape correspondences simultaneously among objects of the same category from only multi-view posed images, without the need of either 3D supervision or ground-truth correspondence knowledge. The learned dense correspondences can be readily used for various image-based tasks such as keypoint detection, part segmentation, and texture transfer that previously require specific model designs. Our method can also accommodate annotation transfer in a one or few-shot manner, given only one or a few instances of the category. Using periodic activation and feature-wise linear modulation (FiLM) conditioning, we introduce deep implicit templates on 3D data into the 3D-aware image synthesis pipeline NeRF. By representing object instances within the same category as shape and appearance variation of a shared NeRF template, our proposed method can achieve dense shape correspondences reasoning on images for a wide range of object classes. We demonstrate the results and applications on both synthetic and real-world data with competitive results compared with other methods based on 3D information.
A Study of the Human Perception of Synthetic Faces
Authors: Bingyu Shen, Brandon RichardWebster, Alice O'Toole, Kevin Bowyer, Walter J. Scheirer
Date: 2021-11-08 00:00:00
Tasks: Face Generation

Advances in face synthesis have raised alarms about the deceptive use of synthetic faces. Can synthetic identities be effectively used to fool human observers? In this paper, we introduce a study of the human perception of synthetic faces generated using different strategies including a state-of-the-art deep learning-based GAN model. This is the first rigorous study of the effectiveness of synthetic face generation techniques grounded in experimental techniques from psychology. We answer important questions such as how often do GAN-based and more traditional image processing-based techniques confuse human observers, and are there subtle cues within a synthetic face image that cause humans to perceive it as a fake without having to search for obvious clues? To answer these questions, we conducted a series of large-scale crowdsourced behavioral experiments with different sources of face imagery. Results show that humans are unable to distinguish synthetic faces from real faces under several different circumstances. This finding has serious implications for many different applications where face images are presented to human users.
Practical, Fast and Robust Point Cloud Registration for 3D Scene Stitching and Object Localization
Authors: Lei Sun
Date: 2021-11-08 00:00:00
Tasks: 3D Feature Matching, Benchmarking, Object Localization, Point Cloud Registration

3D point cloud registration ranks among the most fundamental problems in remote sensing, photogrammetry, robotics and geometric computer vision. Due to the limited accuracy of 3D feature matching techniques, outliers may exist, sometimes even in very large numbers, among the correspondences. Since existing robust solvers may encounter high computational cost or restricted robustness, we propose a novel, fast and highly robust solution, named VOCRA (VOting with Cost function and Rotating Averaging), for the point cloud registration problem with extreme outlier rates. Our first contribution is to employ the Tukey's Biweight robust cost to introduce a new voting and correspondence sorting technique, which proves to be rather effective in distinguishing true inliers from outliers even with extreme (99%) outlier rates. Our second contribution consists in designing a time-efficient consensus maximization paradigm based on robust rotation averaging, serving to seek inlier candidates among the correspondences. Finally, we apply Graduated Non-Convexity with Tukey's Biweight (GNC-TB) to estimate the correct transformation with the inlier candidates obtained, which is then used to find the complete inlier set. Both standard benchmarking and realistic experiments with application to two real-data problems are conducted, and we show that our solver VOCRA is robust against over 99% outliers and more time-efficient than the state-of-the-art competitors.
Rethinking Deconvolution for 2D Human Pose Estimation Light yet Accurate Model for Real-time Edge Computing
Authors: Masayuki Yamazaki, Eigo Mori
Date: 2021-11-08 00:00:00
Tasks: 2D Human Pose Estimation, Decoder, Edge-computing, Pose Estimation, Quantization

In this study, we present a pragmatic lightweight pose estimation model. Our model can achieve real-time predictions using low-power embedded devices. This system was found to be very accurate and achieved a 94.5% accuracy of SOTA HRNet 256x192 using a computational cost of only 3.8% on COCO test dataset. Our model adopts an encoder-decoder architecture and is carefully downsized to improve its efficiency. We especially focused on optimizing the deconvolution layers and observed that the channel reduction of the deconvolution layers contributes significantly to reducing computational resource consumption without degrading the accuracy of this system. We also incorporated recent model agnostic techniques such as DarkPose and distillation training to maximize the efficiency of our model. Furthermore, we applied model quantization to exploit multi/mixed precision features. Our FP16'ed model (COCO AP 70.0) operates at ~60-fps on NVIDIA Jetson AGX Xavier and ~200 fps on NVIDIA Quadro RTX6000.
Global-Local Attention for Emotion Recognition
Authors: Nhat Le, Khanh Nguyen, Anh Nguyen, Bac Le
Date: 2021-11-07 00:00:00
Tasks: Emotion Recognition

Human emotion recognition is an active research area in artificial intelligence and has made substantial progress over the past few years. Many recent works mainly focus on facial regions to infer human affection, while the surrounding context information is not effectively utilized. In this paper, we proposed a new deep network to effectively recognize human emotions using a novel global-local attention mechanism. Our network is designed to extract features from both facial and context regions independently, then learn them together using the attention module. In this way, both the facial and contextual information is used to infer human emotions, therefore enhancing the discrimination of the classifier. The intensive experiments show that our method surpasses the current state-of-the-art methods on recent emotion datasets by a fair margin. Qualitatively, our global-local attention module can extract more meaningful attention maps than previous methods. The source code and trained model of our network are available at https://github.com/minhnhatvt/glamor-net
Hierarchical Segment-based Optimization for SLAM
Authors: Yuxin Tian, Yujie Wang, Ming Ouyang, Xuesong Shi
Date: 2021-11-07 00:00:00
Tasks: Segmentation, Simultaneous Localization and Mapping

This paper presents a hierarchical segment-based optimization method for Simultaneous Localization and Mapping (SLAM) system. First we propose a reliable trajectory segmentation method that can be used to increase efficiency in the back-end optimization. Then we propose a buffer mechanism for the first time to improve the robustness of the segmentation. During the optimization, we use global information to optimize the frames with large error, and interpolation instead of optimization to update well-estimated frames to hierarchically allocate the amount of computation according to error of each frame. Comparative experiments on the benchmark show that our method greatly improves the efficiency of optimization with almost no drop in accuracy, and outperforms existing high-efficiency optimization method by a large margin.
Online Mutual Adaptation of Deep Depth Prediction and Visual SLAM
Authors: Shing Yan Loo, Moein Shakeri, Sai Hong Tang, Syamsiah Mashohor, Hong Zhang
Date: 2021-11-07 00:00:00
Tasks: Depth Estimation, Depth Prediction, Prediction, Simultaneous Localization and Mapping

The ability of accurate depth prediction by a convolutional neural network (CNN) is a major challenge for its wide use in practical visual simultaneous localization and mapping (SLAM) applications, such as enhanced camera tracking and dense mapping. This paper is set out to answer the following question: Can we tune a depth prediction CNN with the help of a visual SLAM algorithm even if the CNN is not trained for the current operating environment in order to benefit the SLAM performance? To this end, we propose a novel online adaptation framework consisting of two complementary processes: a SLAM algorithm that is used to generate keyframes to fine-tune the depth prediction and another algorithm that uses the online adapted depth to improve map quality. Once the potential noisy map points are removed, we perform global photometric bundle adjustment (BA) to improve the overall SLAM performance. Experimental results on both benchmark datasets and a real robot in our own experimental environments show that our proposed method improves the overall SLAM accuracy. While regularization has been shown to be effective in multi-task classification problems, we present experimental results and an ablation study to show the effectiveness of regularization in preventing catastrophic forgetting in the online adaptation of depth prediction, a single-task regression problem. In addition, we compare our online adaptation framework against the state-of-the-art pre-trained depth prediction CNNs to show that our online adapted depth prediction CNN outperforms the depth prediction CNNs that have been trained on a large collection of datasets.
Acquisition-invariant brain MRI segmentation with informative uncertainties
Authors: Pedro Borges, Richard Shaw, Thomas Varsavsky, Kerstin Klaser, David Thomas, Ivana Drobnjak, Sebastien Ourselin, M Jorge Cardoso
Date: 2021-11-07 00:00:00
Tasks: MRI segmentation, Segmentation

Combining multi-site data can strengthen and uncover trends, but is a task that is marred by the influence of site-specific covariates that can bias the data and therefore any downstream analyses. Post-hoc multi-site correction methods exist but have strong assumptions that often do not hold in real-world scenarios. Algorithms should be designed in a way that can account for site-specific effects, such as those that arise from sequence parameter choices, and in instances where generalisation fails, should be able to identify such a failure by means of explicit uncertainty modelling. This body of work showcases such an algorithm, that can become robust to the physics of acquisition in the context of segmentation tasks, while simultaneously modelling uncertainty. We demonstrate that our method not only generalises to complete holdout datasets, preserving segmentation quality, but does so while also accounting for site-specific sequence choices, which also allows it to perform as a harmonisation tool.
Direct Multi-view Multi-person 3D Pose Estimation
Authors: Tao Wang, Jianfeng Zhang, Yujun Cai, Shuicheng Yan, Jiashi Feng
Date: 2021-11-07 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: 3D Multi-Person Pose Estimation, 3D Pose Estimation, Pose Estimation

We present Multi-view Pose transformer (MvP) for estimating multi-person 3D poses from multi-view images. Instead of estimating 3D joint locations from costly volumetric representation or reconstructing the per-person 3D pose from multiple detected 2D poses as in previous methods, MvP directly regresses the multi-person 3D poses in a clean and efficient way, without relying on intermediate tasks. Specifically, MvP represents skeleton joints as learnable query embeddings and let them progressively attend to and reason over the multi-view information from the input images to directly regress the actual 3D joint locations. To improve the accuracy of such a simple pipeline, MvP presents a hierarchical scheme to concisely represent query embeddings of multi-person skeleton joints and introduces an input-dependent query adaptation approach. Further, MvP designs a novel geometrically guided attention mechanism, called projective attention, to more precisely fuse the cross-view information for each joint. MvP also introduces a RayConv operation to integrate the view-dependent camera geometry into the feature representations for augmenting the projective attention. We show experimentally that our MvP model outperforms the state-of-the-art methods on several benchmarks while being much more efficient. Notably, it achieves 92.3% AP25 on the challenging Panoptic dataset, improving upon the previous best approach [36] by 9.8%. MvP is general and also extendable to recovering human mesh represented by the SMPL model, thus useful for modeling multi-person body shapes. Code and models are available at https://github.com/sail-sg/mvp.
Texture-enhanced Light Field Super-resolution with Spatio-Angular Decomposition Kernels
Authors: Zexi Hu, Xiaoming Chen, Henry Wing Fung Yeung, Yuk Ying Chung, Zhibo Chen
Date: 2021-11-07 00:00:00
Tasks: Material Recognition, Super-Resolution

Despite the recent progress in light field super-resolution (LFSR) achieved by convolutional neural networks, the correlation information of light field (LF) images has not been sufficiently studied and exploited due to the complexity of 4D LF data. To cope with such high-dimensional LF data, most of the existing LFSR methods resorted to decomposing it into lower dimensions and subsequently performing optimization on the decomposed sub-spaces. However, these methods are inherently limited as they neglected the characteristics of the decomposition operations and only utilized a limited set of LF sub-spaces ending up failing to sufficiently extract spatio-angular features and leading to a performance bottleneck. To overcome these limitations, in this paper, we comprehensively discover the potentials of LF decomposition and propose a novel concept of decomposition kernels. In particular, we systematically unify the decomposition operations of various sub-spaces into a series of such decomposition kernels, which are incorporated into our proposed Decomposition Kernel Network (DKNet) for comprehensive spatio-angular feature extraction. The proposed DKNet is experimentally verified to achieve considerable improvements compared with the state-of-the-art methods. To further improve DKNet in producing more visually pleasing LFSR results, based on the VGG network, we propose a LFVGG loss to guide the Texture-Enhanced DKNet (TE-DKNet) to generate rich authentic textures and enhance LF images' visual quality significantly. We also propose an indirect evaluation metric by taking advantage of LF material recognition to objectively assess the perceptual enhancement brought by the LFVGG loss.
Can viewer proximity be a behavioural marker for Autism Spectrum Disorder?
Authors: Rahul Bishain, Bhismadev Chakrabarti, Jayashree Dasgupta, Indu Dubey, Sharat Chandran
Date: 2021-11-07 00:00:00

Screening for any of the Autism Spectrum Disorders is a complicated process often involving a hybrid of behavioural observations and questionnaire based tests. Typically carried out in a controlled setting, this process requires trained clinicians or psychiatrists for such assessments. Riding on the wave of technical advancement in mobile platforms, several attempts have been made at incorporating such assessments on mobile and tablet devices. In this paper we analyse videos generated using one such screening test. This paper reports the first use of the efficacy of using the observer's distance from the display screen while administering a sensory sensitivity test as a behavioural marker for autism for children aged 2-7 years The potential for using a test such as this in casual home settings is promising.
Are we ready for a new paradigm shift? A Survey on Visual Deep MLP
Authors: Ruiyang Liu, Yinghui Li, Linmi Tao, Dun Liang, Hai-Tao Zheng
Date: 2021-11-07 00:00:00
Tasks: GPU

Recently, the proposed deep MLP models have stirred up a lot of interest in the vision community. Historically, the availability of larger datasets combined with increased computing capacity leads to paradigm shifts. This review paper provides detailed discussions on whether MLP can be a new paradigm for computer vision. We compare the intrinsic connections and differences between convolution, self-attention mechanism, and Token-mixing MLP in detail. Advantages and limitations of Token-mixing MLP are provided, followed by careful analysis of recent MLP-like variants, from module design to network architecture, and their applications. In the GPU era, the locally and globally weighted summations are the current mainstreams, represented by the convolution and self-attention mechanism, as well as MLP. We suggest the further development of paradigm to be considered alongside the next-generation computing devices.
Registration Techniques for Deformable Objects
Authors: Alireza Ahmadi
Date: 2021-11-07 00:00:00

In general, the problem of non-rigid registration is about matching two different scans of a dynamic object taken at two different points in time. These scans can undergo both rigid motions and non-rigid deformations. Since new parts of the model may come into view and other parts get occluded in between two scans, the region of overlap is a subset of both scans. In the most general setting, no prior template shape is given and no markers or explicit feature point correspondences are available. So, this case is a partial matching problem that takes into account the assumption that consequent scans undergo small deformations while having a significant amount of overlapping area [28]. The problem which this thesis is addressing is mapping deforming objects and localizing cameras in the environment at the same time.
Information Extraction from Visually Rich Documents with Font Style Embeddings
Authors: Ismail Oussaid, William Vanhuffel, Pirashanth Ratnamogan, Mhamed Hajaiej, Alexis Mathey, Thomas Gilles
Date: 2021-11-07 00:00:00

Information extraction (IE) from documents is an intensive area of research with a large set of industrial applications. Current state-of-the-art methods focus on scanned documents with approaches combining computer vision, natural language processing and layout representation. We propose to challenge the usage of computer vision in the case where both token style and visual representation are available (i.e native PDF documents). Our experiments on three real-world complex datasets demonstrate that using token style attributes based embedding instead of a raw visual embedding in LayoutLM model is beneficial. Depending on the dataset, such an embedding yields an improvement of 0.18% to 2.29% in the weighted F1-score with a decrease of 30.7% in the final number of trainable parameters of the model, leading to an improvement in both efficiency and effectiveness.