pwc-archive/papers-with-abstracts huggingface dataset:
An exploration of the performances achievable by combining unsupervised background subtraction algorithms
Authors: Sébastien Piérard, Marc Braham, Marc Van Droogenbroeck
Date: 2022-02-25 00:00:00
Tasks: Motion Detection

Background subtraction (BGS) is a common choice for performing motion detection in video. Hundreds of BGS algorithms are released every year, but combining them to detect motion remains largely unexplored. We found that combination strategies allow to capitalize on this massive amount of available BGS algorithms, and offer significant space for performance improvement. In this paper, we explore sets of performances achievable by 6 strategies combining, pixelwise, the outputs of 26 unsupervised BGS algorithms, on the CDnet 2014 dataset, both in the ROC space and in terms of the F1 score. The chosen strategies are representative for a large panel of strategies, including both deterministic and non-deterministic ones, voting and learning. In our experiments, we compare our results with the state-of-the-art combinations IUTIS-5 and CNN-SFC, and report six conclusions, among which the existence of an important gap between the performances of the individual algorithms and the best performances achievable by combining them.
Active Learning for Point Cloud Semantic Segmentation via Spatial-Structural Diversity Reasoning
Authors: Feifei Shao, Yawei Luo, Ping Liu, Jie Chen, Yi Yang, Yulei Lu, Jun Xiao
Date: 2022-02-25 00:00:00
Tasks: Active Learning, Diversity, Semantic Segmentation

The expensive annotation cost is notoriously known as the main constraint for the development of the point cloud semantic segmentation technique. Active learning methods endeavor to reduce such cost by selecting and labeling only a subset of the point clouds, yet previous attempts ignore the spatial-structural diversity of the selected samples, inducing the model to select clustered candidates with similar shapes in a local area while missing other representative ones in the global environment. In this paper, we propose a new 3D region-based active learning method to tackle this problem. Dubbed SSDR-AL, our method groups the original point clouds into superpoints and incrementally selects the most informative and representative ones for label acquisition. We achieve the selection mechanism via a graph reasoning network that considers both the spatial and structural diversities of superpoints. To deploy SSDR-AL in a more practical scenario, we design a noise-aware iterative labeling strategy to confront the "noisy annotation" problem introduced by the previous "dominant labeling" strategy in superpoints. Extensive experiments on two point cloud benchmarks demonstrate the effectiveness of SSDR-AL in the semantic segmentation task. Particularly, SSDR-AL significantly outperforms the baseline method and reduces the annotation cost by up to 63.0% and 24.0% when achieving 90% performance of fully supervised learning, respectively.
LF-VIO: A Visual-Inertial-Odometry Framework for Large Field-of-View Cameras with Negative Plane
Authors: Ze Wang, Kailun Yang, Hao Shi, Peng Li, Fei Gao, Kaiwei Wang
Date: 2022-02-25 00:00:00
Tasks: Autonomous Driving, Visual Odometry

Visual-inertial-odometry has attracted extensive attention in the field of autonomous driving and robotics. The size of Field of View (FoV) plays an important role in Visual-Odometry (VO) and Visual-Inertial-Odometry (VIO), as a large FoV enables to perceive a wide range of surrounding scene elements and features. However, when the field of the camera reaches the negative half plane, one cannot simply use [u,v,1]^T to represent the image feature points anymore. To tackle this issue, we propose LF-VIO, a real-time VIO framework for cameras with extremely large FoV. We leverage a three-dimensional vector with unit length to represent feature points, and design a series of algorithms to overcome this challenge. To address the scarcity of panoramic visual odometry datasets with ground-truth location and pose, we present the PALVIO dataset, collected with a Panoramic Annular Lens (PAL) system with an entire FoV of 360{\deg}x(40{\deg}-120{\deg}) and an IMU sensor. With a comprehensive variety of experiments, the proposed LF-VIO is verified on both the established PALVIO benchmark and a public fisheye camera dataset with a FoV of 360{\deg}x(0{\deg}-93.5{\deg}). LF-VIO outperforms state-of-the-art visual-inertial-odometry methods. Our dataset and code are made publicly available at https://github.com/flysoaryun/LF-VIO
Joint Answering and Explanation for Visual Commonsense Reasoning
Authors: Zhenyang Li, Yangyang Guo, Kejie Wang, Yinwei Wei, Liqiang Nie, Mohan Kankanhalli
Date: 2022-02-25 00:00:00
Tasks: Knowledge Distillation, Question Answering, Visual Commonsense Reasoning, Visual Question Answering, Visual Question Answering (VQA), Visual Reasoning

Visual Commonsense Reasoning (VCR), deemed as one challenging extension of the Visual Question Answering (VQA), endeavors to pursue a more high-level visual comprehension. It is composed of two indispensable processes: question answering over a given image and rationale inference for answer explanation. Over the years, a variety of methods tackling VCR have advanced the performance on the benchmark dataset. Despite significant as these methods are, they often treat the two processes in a separate manner and hence decompose the VCR into two irrelevant VQA instances. As a result, the pivotal connection between question answering and rationale inference is interrupted, rendering existing efforts less faithful on visual reasoning. To empirically study this issue, we perform some in-depth explorations in terms of both language shortcuts and generalization capability to verify the pitfalls of this treatment. Based on our findings, in this paper, we present a plug-and-play knowledge distillation enhanced framework to couple the question answering and rationale inference processes. The key contribution is the introduction of a novel branch, which serves as the bridge to conduct processes connecting. Given that our framework is model-agnostic, we apply it to the existing popular baselines and validate its effectiveness on the benchmark dataset. As detailed in the experimental results, when equipped with our framework, these baselines achieve consistent and significant performance improvements, demonstrating the viability of processes coupling, as well as the superiority of the proposed framework.
Deep Dirichlet uncertainty for unsupervised out-of-distribution detection of eye fundus photographs in glaucoma screening
Authors: Teresa Araújo, Guilherme Aresta, Hrvoje Bogunovic
Date: 2022-02-25 00:00:00
Tasks: Out-of-Distribution Detection

The development of automatic tools for early glaucoma diagnosis with color fundus photographs can significantly reduce the impact of this disease. However, current state-of-the-art solutions are not robust to real-world scenarios, providing over-confident predictions for out-of-distribution cases. With this in mind, we propose a model based on the Dirichlet distribution that allows to obtain class-wise probabilities together with an uncertainty estimation without exposure to out-of-distribution cases. We demonstrate our approach on the AIROGS challenge. At the start of the final test phase (8 Feb. 2022), our method had the highest average score among all submissions.
Improving Amharic Handwritten Word Recognition Using Auxiliary Task
Authors: Mesay Samuel Gondere, Lars Schmidt-Thieme, Durga Prasad Sharma, Abiot Sinamo Boltena
Date: 2022-02-25 00:00:00
Tasks: Handwritten Text Recognition, Optical Character Recognition, Optical Character Recognition (OCR)

Amharic is one of the official languages of the Federal Democratic Republic of Ethiopia. It is one of the languages that use an Ethiopic script which is derived from Gee'z, ancient and currently a liturgical language. Amharic is also one of the most widely used literature-rich languages of Ethiopia. There are very limited innovative and customized research works in Amharic optical character recognition (OCR) in general and Amharic handwritten text recognition in particular. In this study, Amharic handwritten word recognition will be investigated. State-of-the-art deep learning techniques including convolutional neural networks together with recurrent neural networks and connectionist temporal classification (CTC) loss were used to make the recognition in an end-to-end fashion. More importantly, an innovative way of complementing the loss function using the auxiliary task from the row-wise similarities of the Amharic alphabet was tested to show a significant recognition improvement over a baseline method. Such findings will promote innovative problem-specific solutions as well as will open insight to a generalized solution that emerges from problem-specific domains.
On Modality Bias Recognition and Reduction
Authors: Yangyang Guo, Liqiang Nie, Harry Cheng, Zhiyong Cheng, Mohan Kankanhalli, Alberto del Bimbo
Date: 2022-02-25 00:00:00
Tasks: Action Recognition, Multi-modal Classification, Question Answering, Temporal Action Localization, Visual Question Answering, Visual Question Answering (VQA)

Making each modality in multi-modal data contribute is of vital importance to learning a versatile multi-modal model. Existing methods, however, are often dominated by one or few of modalities during model training, resulting in sub-optimal performance. In this paper, we refer to this problem as modality bias and attempt to study it in the context of multi-modal classification systematically and comprehensively. After stepping into several empirical analysis, we recognize that one modality affects the model prediction more just because this modality has a spurious correlation with instance labels. In order to primarily facilitate the evaluation on the modality bias problem, we construct two datasets respectively for the colored digit recognition and video action recognition tasks in line with the Out-of-Distribution (OoD) protocol. Collaborating with the benchmarks in the visual question answering task, we empirically justify the performance degradation of the existing methods on these OoD datasets, which serves as evidence to justify the modality bias learning. In addition, to overcome this problem, we propose a plug-and-play loss function method, whereby the feature space for each label is adaptively learned according to the training set statistics. Thereafter, we apply this method on eight baselines in total to test its effectiveness. From the results on four datasets regarding the above three tasks, our method yields remarkable performance improvements compared with the baselines, demonstrating its superiority on reducing the modality bias problem.
Reconstruction of Perceived Images from fMRI Patterns and Semantic Brain Exploration using Instance-Conditioned GANs
Authors: Furkan Ozcelik, Bhavin Choksi, Milad Mozafari, Leila Reddy, Rufin VanRullen
Date: 2022-02-25 00:00:00
Tasks: Image Generation, Playing the Game of 2048, regression, Self-Supervised Learning

Reconstructing perceived natural images from fMRI signals is one of the most engaging topics of neural decoding research. Prior studies had success in reconstructing either the low-level image features or the semantic/high-level aspects, but rarely both. In this study, we utilized an Instance-Conditioned GAN (IC-GAN) model to reconstruct images from fMRI patterns with both accurate semantic attributes and preserved low-level details. The IC-GAN model takes as input a 119-dim noise vector and a 2048-dim instance feature vector extracted from a target image via a self-supervised learning model (SwAV ResNet-50); these instance features act as a conditioning for IC-GAN image generation, while the noise vector introduces variability between samples. We trained ridge regression models to predict instance features, noise vectors, and dense vectors (the output of the first dense layer of the IC-GAN generator) of stimuli from corresponding fMRI patterns. Then, we used the IC-GAN generator to reconstruct novel test images based on these fMRI-predicted variables. The generated images presented state-of-the-art results in terms of capturing the semantic attributes of the original test images while remaining relatively faithful to low-level image details. Finally, we use the learned regression model and the IC-GAN generator to systematically explore and visualize the semantic features that maximally drive each of several regions-of-interest in the human brain.
The effect of fatigue on the performance of online writer recognition
Authors: Enric Sesa-Nogueras, Marcos Faundez-Zanuy, Manuel-Vicente Garnacho-Castaño
Date: 2022-02-24 00:00:00
Tasks: Dynamic Time Warping, Quantization

Background: The performance of biometric modalities based on things done by the subject, like signature and text-based recognition, may be affected by the subject state. Fatigue is one of the conditions that can significantly affect the outcome of handwriting tasks. Recent research has already shown that physical fatigue produces measurable differences in some features extracted from common writing and drawing tasks. It is important to establish to which extent physical fatigue contributes to the intra-person variability observed in these biometric modalities and also to know whether the performance of recognition methods is affected by fatigue. Goal: In this paper we assess the impact of fatigue on intra-user variability and on the performance of signature-based and text-based writer recognition approaches encompassing both identification and verification. Methods: Several signature and text recognition methods are considered and applied to samples gathered after different levels of induced fatigue, measured by metabolic and mechanical assessment and, also by subjective perception. The recognition methods are Dynamic Time Warping and Multi Section Vector Quantization, for signatures, and Allographic Text-Dependent Recognition for text in capital letters. For each fatigue level, the identification and verification performance of these methods is measured. Results: Signature shows no statistically significant intra-user impact, but text does. On the other hand, performance of signature-based recognition approaches is negatively impacted by fatigue whereas the impact is not noticeable in text-based recognition, provided long enough sequences are considered.
Synthesizing Photorealistic Images with Deep Generative Learning
Authors: Chuanxia Zheng
Date: 2022-02-23 00:00:00
Tasks: Depth Estimation, Image Generation, Translation

The goal of this thesis is to present my research contributions towards solving various visual synthesis and generation tasks, comprising image translation, image completion, and completed scene decomposition. This thesis consists of five pieces of work, each of which presents a new learning-based approach for synthesizing images with plausible content as well as visually realistic appearance. Each work demonstrates the superiority of the proposed approach on image synthesis, with some further contributing to other tasks, such as depth estimation.
Towards Safe, Real-Time Systems: Stereo vs Images and LiDAR for 3D Object Detection
Authors: Matthew Levine
Date: 2022-02-25 00:00:00
Tasks: 3D Object Detection, object-detection, Object Detection

As object detectors rapidly improve, attention has expanded past image-only networks to include a range of 3D and multimodal frameworks, especially ones that incorporate LiDAR. However, due to cost, logistics, and even some safety considerations, stereo can be an appealing alternative. Towards understanding the efficacy of stereo as a replacement for monocular input or LiDAR in object detectors, we show that multimodal learning with traditional disparity algorithms can improve image-based results without increasing the number of parameters, and that learning over stereo error can impart similar 3D localization power to LiDAR in certain contexts. Furthermore, doing so also has calibration benefits with respect to image-only methods. We benchmark on the public dataset KITTI, and in doing so, reveal a few small but common algorithmic mistakes currently used in computing metrics on that set, and offer efficient, provably correct alternatives.
Sensing accident-prone features in urban scenes for proactive driving and accident prevention
Authors: Sumit Mishra, Praveen Kumar Rajendran, Luiz Felipe Vecchietti, Dongsoo Har
Date: 2022-02-25 00:00:00

In urban cities, visual information on and along roadways is likely to distract drivers and lead to missing traffic signs and other accident-prone (AP) features. To avoid accidents due to missing these visual cues, this paper proposes a visual notification of AP-features to drivers based on real-time images obtained via dashcam. For this purpose, Google Street View images around accident hotspots (areas of dense accident occurrence) identified by a real-accident dataset are used to train a novel attention module to classify a given urban scene into an accident hotspot or a non-hotspot (area of sparse accident occurrence). The proposed module leverages channel, point, and spatial-wise attention learning on top of different CNN backbones. This leads to better classification results and more certain AP-features with better contextual knowledge when compared with CNN backbones alone. Our proposed module achieves up to 92% classification accuracy. The capability of detecting AP-features by the proposed model were analyzed by a comparative study of three different class activation map (CAM) methods, which were used to inspect specific AP-features causing the classification decision. Outputs of CAM methods were processed by an image processing pipeline to extract only the AP-features that are explainable to drivers and notified using a visual notification system. Range of experiments was performed to prove the efficacy and AP-features of the system. Ablation of the AP-features taking 9.61%, on average, of the total area in each image increased the chance of a given area to be classified as a non-hotspot by up to 21.8%.
NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions
Authors: Yuheng Jiang, Suyi Jiang, Guoxing Sun, Zhuo Su, Kaiwen Guo, Minye Wu, Jingyi Yu, Lan Xu
Date: 2022-02-25 00:00:00
Proceeding: CVPR 2022 1
Tasks: Human-Object Interaction Detection, Multi-Task Learning, Object, Object Tracking

4D modeling of human-object interactions is critical for numerous applications. However, efficient volumetric capture and rendering of complex interaction scenarios, especially from sparse inputs, remain challenging. In this paper, we propose NeuralHOFusion, a neural approach for volumetric human-object capture and rendering using sparse consumer RGBD sensors. It marries traditional non-rigid fusion with recent neural implicit modeling and blending advances, where the captured humans and objects are layerwise disentangled. For geometry modeling, we propose a neural implicit inference scheme with non-rigid key-volume fusion, as well as a template-aid robust object tracking pipeline. Our scheme enables detailed and complete geometry generation under complex interactions and occlusions. Moreover, we introduce a layer-wise human-object texture rendering scheme, which combines volumetric and image-based rendering in both spatial and temporal domains to obtain photo-realistic results. Extensive experiments demonstrate the effectiveness and efficiency of our approach in synthesizing photo-realistic free-view results under complex human-object interactions.
RelMobNet: End-to-end relative camera pose estimation using a robust two-stage training
Authors: Praveen Kumar Rajendran, Sumit Mishra, Luiz Felipe Vecchietti, Dongsoo Har
Date: 2022-02-25 00:00:00
Tasks: 3D Reconstruction, Camera Pose Estimation, Pose Estimation, Translation, Visual Localization

Relative camera pose estimation, i.e. estimating the translation and rotation vectors using a pair of images taken in different locations, is an important part of systems in augmented reality and robotics. In this paper, we present an end-to-end relative camera pose estimation network using a siamese architecture that is independent of camera parameters. The network is trained using the Cambridge Landmarks data with four individual scene datasets and a dataset combining the four scenes. To improve generalization, we propose a novel two-stage training that alleviates the need of a hyperparameter to balance the translation and rotation loss scale. The proposed method is compared with one-stage training CNN-based methods such as RPNet and RCPNet and demonstrate that the proposed model improves translation vector estimation by 16.11%, 28.88%, and 52.27% on the Kings College, Old Hospital, and St Marys Church scenes, respectively. For proving texture invariance, we investigate the generalization of the proposed method augmenting the datasets to different scene styles, as ablation studies, using generative adversarial networks. Also, we present a qualitative assessment of epipolar lines of our network predictions and ground truth poses.
ARIA: Adversarially Robust Image Attribution for Content Provenance
Authors: Maksym Andriushchenko, Xiaoyang Rebecca Li, Geoffrey Oxholm, Thomas Gittings, Tu Bui, Nicolas Flammarion, John Collomosse
Date: 2022-02-25 00:00:00
Tasks: Contrastive Learning, Image Attribution, Misinformation, valid

Image attribution -- matching an image back to a trusted source -- is an emerging tool in the fight against online misinformation. Deep visual fingerprinting models have recently been explored for this purpose. However, they are not robust to tiny input perturbations known as adversarial examples. First we illustrate how to generate valid adversarial images that can easily cause incorrect image attribution. Then we describe an approach to prevent imperceptible adversarial attacks on deep visual fingerprinting models, via robust contrastive learning. The proposed training procedure leverages training on $\ell_\infty$-bounded adversarial examples, it is conceptually simple and incurs only a small computational overhead. The resulting models are substantially more robust, are accurate even on unperturbed images, and perform well even over a database with millions of images. In particular, we achieve 91.6% standard and 85.1% adversarial recall under $\ell_\infty$-bounded perturbations on manipulated images compared to 80.1% and 0.0% from prior work. We also show that robustness generalizes to other types of imperceptible perturbations unseen during training. Finally, we show how to train an adversarially robust image comparator model for detecting editorial changes in matched images.
Time Efficient Training of Progressive Generative Adversarial Network using Depthwise Separable Convolution and Super Resolution Generative Adversarial Network
Authors: Atharva Karwande, Pranesh Kulkarni, Tejas Kolhe, Akshay Joshi, Soham Kamble
Date: 2022-02-24 00:00:00
Tasks: Generative Adversarial Network, Image Augmentation, Super-Resolution

Generative Adversarial Networks have been employed successfully to generate high-resolution augmented images of size 1024^2. Although the augmented images generated are unprecedented, the training time of the model is exceptionally high. Conventional GAN requires training of both Discriminator as well as the Generator. In Progressive GAN, which is the current state-of-the-art GAN for image augmentation, instead of training the GAN all at once, a new concept of progressing growing of Discriminator and Generator simultaneously, was proposed. Although the lower stages such as 4x4 and 8x8 train rather quickly, the later stages consume a tremendous amount of time which could take days to finish the model training. In our paper, we propose a novel pipeline that combines Progressive GAN with slight modifications and Super Resolution GAN. Super Resolution GAN up samples low-resolution images to high-resolution images which can prove to be a useful resource to reduce the training time exponentially.