pwc-archive/papers-with-abstracts huggingface dataset:
Arbitrary Shape Text Detection via Boundary Transformer
Authors: Shi-Xue Zhang, Chun Yang, Xiaobin Zhu, Xu-Cheng Yin
Date: 2022-05-11 00:00:00
Tasks: Decoder, Text Detection

In arbitrary shape text detection, locating accurate text boundaries is challenging and non-trivial. Existing methods often suffer from indirect text boundary modeling or complex post-processing. In this paper, we systematically present a unified coarse-to-fine framework via boundary learning for arbitrary shape text detection, which can accurately and efficiently locate text boundaries without post-processing. In our method, we explicitly model the text boundary via an innovative iterative boundary transformer in a coarse-to-fine manner. In this way, our method can directly gain accurate text boundaries and abandon complex post-processing to improve efficiency. Specifically, our method mainly consists of a feature extraction backbone, a boundary proposal module, and an iteratively optimized boundary transformer module. The boundary proposal module consisting of multi-layer dilated convolutions will compute important prior information (including classification map, distance field, and direction field) for generating coarse boundary proposals while guiding the boundary transformer's optimization. The boundary transformer module adopts an encoder-decoder structure, in which the encoder is constructed by multi-layer transformer blocks with residual connection while the decoder is a simple multi-layer perceptron network (MLP). Under the guidance of prior information, the boundary transformer module will gradually refine the coarse boundary proposals via iterative boundary deformation. Furthermore, we propose a novel boundary energy loss (BEL) which introduces an energy minimization constraint and an energy monotonically decreasing constraint to further optimize and stabilize the learning of boundary refinement. Extensive experiments on publicly available and challenging datasets demonstrate the state-of-the-art performance and promising efficiency of our method.
Deep Depth Completion from Extremely Sparse Data: A Survey
Authors: Junjie Hu, Chenyu Bao, Mete Ozay, Chenyou Fan, Qing Gao, Honghai Liu, Tin Lun Lam
Date: 2022-05-11 00:00:00
Tasks: 3D Reconstruction, Autonomous Driving, Depth Completion, Robot Navigation, Survey

Depth completion aims at predicting dense pixel-wise depth from an extremely sparse map captured from a depth sensor, e.g., LiDARs. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Recent successes on the task have been demonstrated and dominated by deep learning based solutions. In this article, for the first time, we provide a comprehensive literature review that helps readers better grasp the research trends and clearly understand the current advances. We investigate the related studies from the design aspects of network architectures, loss functions, benchmark datasets, and learning strategies with a proposal of a novel taxonomy that categorizes existing methods. Besides, we present a quantitative comparison of model performance on three widely used benchmarks, including indoor and outdoor datasets. Finally, we discuss the challenges of prior works and provide readers with some insights for future research directions.
AutoLC: Search Lightweight and Top-Performing Architecture for Remote Sensing Image Land-Cover Classification
Authors: Chenyu Zheng, Junjue Wang, Ailong Ma, Yanfei Zhong
Date: 2022-05-11 00:00:00
Tasks: Decoder, Land Cover Classification, Neural Architecture Search

Land-cover classification has long been a hot and difficult challenge in remote sensing community. With massive High-resolution Remote Sensing (HRS) images available, manually and automatically designed Convolutional Neural Networks (CNNs) have already shown their great latent capacity on HRS land-cover classification in recent years. Especially, the former can achieve better performance while the latter is able to generate lightweight architecture. Unfortunately, they both have shortcomings. On the one hand, because manual CNNs are almost proposed for natural image processing, it becomes very redundant and inefficient to process HRS images. On the other hand, nascent Neural Architecture Search (NAS) techniques for dense prediction tasks are mainly based on encoder-decoder architecture, and just focus on the automatic design of the encoder, which makes it still difficult to recover the refined mapping when confronting complicated HRS scenes. To overcome their defects and tackle the HRS land-cover classification problems better, we propose AutoLC which combines the advantages of two methods. First, we devise a hierarchical search space and gain the lightweight encoder underlying gradient-based search strategy. Second, we meticulously design a lightweight but top-performing decoder that is adaptive to the searched encoder of itself. Finally, experimental results on the LoveDA land-cover dataset demonstrate that our AutoLC method outperforms the state-of-art manual and automatic methods with much less computational consumption.
Recurrent Encoder-Decoder Networks for Vessel Trajectory Prediction with Uncertainty Estimation
Authors: Samuele Capobianco, Nicola Forti, Leonardo M. Millefiori, Paolo Braca, Peter Willett
Date: 2022-05-11 00:00:00
Tasks: Decoder, Deep Learning, Prediction, Trajectory Prediction, Uncertainty Quantification

Recent deep learning methods for vessel trajectory prediction are able to learn complex maritime patterns from historical Automatic Identification System (AIS) data and accurately predict sequences of future vessel positions with a prediction horizon of several hours. However, in maritime surveillance applications, reliably quantifying the prediction uncertainty can be as important as obtaining high accuracy. This paper extends deep learning frameworks for trajectory prediction tasks by exploring how recurrent encoder-decoder neural networks can be tasked not only to predict but also to yield a corresponding prediction uncertainty via Bayesian modeling of epistemic and aleatoric uncertainties. We compare the prediction performance of two different models based on labeled or unlabeled input data to highlight how uncertainty quantification and accuracy can be improved by using, if available, additional information on the intention of the ship (e.g., its planned destination).
An Objective Method for Pedestrian Occlusion Level Classification
Authors: Shane Gilroy, Martin Glavin, Edward Jones, Darragh Mullins
Date: 2022-05-11 00:00:00
Tasks: Autonomous Vehicles, Classification, Pedestrian Detection

Pedestrian detection is among the most safety-critical features of driver assistance systems for autonomous vehicles. One of the most complex detection challenges is that of partial occlusion, where a target object is only partially available to the sensor due to obstruction by another foreground object. A number of current pedestrian detection benchmarks provide annotation for partial occlusion to assess algorithm performance in these scenarios, however each benchmark varies greatly in their definition of the occurrence and severity of occlusion. In addition, current occlusion level annotation methods contain a high degree of subjectivity by the human annotator. This can lead to inaccurate or inconsistent reporting of an algorithm's detection performance for partially occluded pedestrians, depending on which benchmark is used. This research presents a novel, objective method for pedestrian occlusion level classification for ground truth annotation. Occlusion level classification is achieved through the identification of visible pedestrian keypoints and through the use of a novel, effective method of 2D body surface area estimation. Experimental results demonstrate that the proposed method reflects the pixel-wise occlusion level of pedestrians in images and is effective for all forms of occlusion, including challenging edge cases such as self-occlusion, truncation and inter-occluding pedestrians.
Multi-Label Logo Recognition and Retrieval based on Weighted Fusion of Neural Features
Authors: Marisa Bernabeu, Antonio Javier Gallego, Antonio Pertusa
Date: 2022-05-11 00:00:00
Tasks: Classification, Image Classification, Image Retrieval, Logo Recognition, Multi-Label Classification, Retrieval

Classifying logo images is a challenging task as they contain elements such as text or shapes that can represent anything from known objects to abstract shapes. While the current state of the art for logo classification addresses the problem as a multi-class task focusing on a single characteristic, logos can have several simultaneous labels, such as different colors. This work proposes a method that allows visually similar logos to be classified and searched from a set of data according to their shape, color, commercial sector, semantics, general characteristics, or a combination of features selected by the user. Unlike previous approaches, the proposal employs a series of multi-label deep neural networks specialized in specific attributes and combines the obtained features to perform the similarity search. To delve into the classification system, different existing logo topologies are compared and some of their problems are analyzed, such as the incomplete labeling that trademark registration databases usually contain. The proposal is evaluated considering 76,000 logos (7 times more than previous approaches) from the European Union Trademarks dataset, which is organized hierarchically using the Vienna ontology. Overall, experimentation attains reliable quantitative and qualitative results, reducing the normalized average rank error of the state-of-the-art from 0.040 to 0.018 for the Trademark Image Retrieval task. Finally, given that the semantics of logos can often be subjective, graphic design students and professionals were surveyed. Results show that the proposed methodology provides better labeling than a human expert operator, improving the label ranking average precision from 0.53 to 0.68.
RustSEG -- Automated segmentation of corrosion using deep learning
Authors: B. Burton, W. T. Nash, N. Birbilis
Date: 2022-05-11 00:00:00
Tasks: Deep Learning, Segmentation

The inspection of infrastructure for corrosion remains a task that is typically performed manually by qualified engineers or inspectors. This task of inspection is laborious, slow, and often requires complex access. Recently, deep learning based algorithms have revealed promise and performance in the automatic detection of corrosion. However, to date, research regarding the segmentation of images for automated corrosion detection has been limited, due to the lack of availability of per-pixel labelled data sets which are required for model training. Herein, a novel deep learning approach (termed RustSEG) is presented, that can accurately segment images for automated corrosion detection, without the requirement of per-pixel labelled data sets for training. The RustSEG method will first, using deep learning techniques, determine if corrosion is present in an image (i.e. a classification task), and then if corrosion is present, the model will examine what pixels in the original image contributed to that classification decision. Finally, the method can refine its predictions into a pixel-level segmentation mask. In ideal cases, the method is able to generate precise masks of corrosion in images, demonstrating that the automated segmentation of corrosion without per-pixel training data is possible, addressing a significant hurdle in automated infrastructure inspection.
A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials
Authors: Chuqiao Li, Zhiwu Huang, Danda Pani Paudel, Yabin Wang, Mohamad Shahbazi, Xiaopeng Hong, Luc van Gool
Date: 2022-05-11 00:00:00
Tasks: Continual Learning, DeepFake Detection, Face Swapping, Incremental Learning

There have been emerging a number of benchmarks and techniques for the detection of deepfakes. However, very few works study the detection of incrementally appearing deepfakes in the real-world scenarios. To simulate the wild scenes, this paper suggests a continual deepfake detection benchmark (CDDB) over a new collection of deepfakes from both known and unknown generative models. The suggested CDDB designs multiple evaluations on the detection over easy, hard, and long sequence of deepfake tasks, with a set of appropriate measures. In addition, we exploit multiple approaches to adapt multiclass incremental learning methods, commonly used in the continual visual recognition, to the continual deepfake detection problem. We evaluate existing methods, including their adapted ones, on the proposed CDDB. Within the proposed benchmark, we explore some commonly known essentials of standard continual learning. Our study provides new insights on these essentials in the context of continual deepfake detection. The suggested CDDB is clearly more challenging than the existing benchmarks, which thus offers a suitable evaluation avenue to the future research. Both data and code are available at https://github.com/Coral79/CDDB.
Contrastive Supervised Distillation for Continual Representation Learning
Authors: Tommaso Barletti, Niccolo' Biondi, Federico Pernici, Matteo Bruni, Alberto del Bimbo
Date: 2022-05-11 00:00:00
Tasks: Representation Learning, Retrieval

In this paper, we propose a novel training procedure for the continual representation learning problem in which a neural network model is sequentially learned to alleviate catastrophic forgetting in visual search tasks. Our method, called Contrastive Supervised Distillation (CSD), reduces feature forgetting while learning discriminative features. This is achieved by leveraging labels information in a distillation setting in which the student model is contrastively learned from the teacher model. Extensive experiments show that CSD performs favorably in mitigating catastrophic forgetting by outperforming current state-of-the-art methods. Our results also provide further evidence that feature forgetting evaluated in visual retrieval tasks is not as catastrophic as in classification tasks. Code at: https://github.com/NiccoBiondi/ContrastiveSupervisedDistillation.
Scene Consistency Representation Learning for Video Scene Segmentation
Authors: Haoqian Wu, Keyu Chen, Yanan Luo, Ruizhi Qiao, Bo Ren, Haozhe Liu, Weicheng Xie, Linlin Shen
Date: 2022-05-11 00:00:00
Proceeding: CVPR 2022 1
Tasks: Data Augmentation, Inductive Bias, Representation Learning, Scene Segmentation, Self-Supervised Learning

A long-term video, such as a movie or TV show, is composed of various scenes, each of which represents a series of shots sharing the same semantic story. Spotting the correct scene boundary from the long-term video is a challenging task, since a model must understand the storyline of the video to figure out where a scene starts and ends. To this end, we propose an effective Self-Supervised Learning (SSL) framework to learn better shot representations from unlabeled long-term videos. More specifically, we present an SSL scheme to achieve scene consistency, while exploring considerable data augmentation and shuffling methods to boost the model generalizability. Instead of explicitly learning the scene boundary features as in the previous methods, we introduce a vanilla temporal model with less inductive bias to verify the quality of the shot features. Our method achieves the state-of-the-art performance on the task of Video Scene Segmentation. Additionally, we suggest a more fair and reasonable benchmark to evaluate the performance of Video Scene Segmentation methods. The code is made available.
Deep Learning and Computer Vision Techniques for Microcirculation Analysis: A Review
Authors: Maged Abdalla Helmy Mohamed Abdou, Trung Tuyen Truong, Eric Jul, Paulo Ferreira
Date: 2022-05-11 00:00:00
Tasks: Survey

The analysis of microcirculation images has the potential to reveal early signs of life-threatening diseases like sepsis. Quantifying the capillary density and the capillary distribution in microcirculation images can be used as a biological marker to assist critically ill patients. The quantification of these biological markers is labor-intensive, time-consuming, and subject to interobserver variability. Several computer vision techniques with varying performance can be used to automate the analysis of these microcirculation images in light of the stated challenges. In this paper, we present a survey of over 50 research papers and present the most relevant and promising computer vision algorithms to automate the analysis of microcirculation images. Furthermore, we present a survey of the methods currently used by other researchers to automate the analysis of microcirculation images. This survey is of high clinical relevance because it acts as a guidebook of techniques for other researchers to develop their microcirculation analysis systems and algorithms.
TextMatcher: Cross-Attentional Neural Network to Compare Image and Text
Authors: Valentina Arrigoni, Luisa Repele, Dario Marino Saccavino
Date: 2022-05-11 00:00:00
Tasks: Text Matching

We study a novel multimodal-learning problem, which we call text matching: given an image containing a single-line text and a candidate text transcription, the goal is to assess whether the text represented in the image corresponds to the candidate text. We devise the first machine-learning model specifically designed for this problem. The proposed model, termed TextMatcher, compares the two inputs by applying a cross-attention mechanism over the embedding representations of image and text, and it is trained in an end-to-end fashion. We extensively evaluate the empirical performance of TextMatcher on the popular IAM dataset. Results attest that, compared to a baseline and existing models designed for related problems, TextMatcher achieves higher performance on a variety of configurations, while at the same time running faster at inference time. We also showcase TextMatcher in a real-world application scenario concerning the automatic processing of bank cheques.
READ: Large-Scale Neural Scene Rendering for Autonomous Driving
Authors: Zhuopeng Li, Lu Li, Zeyu Ma, Ping Zhang, Junbo Chen, Jianke Zhu
Date: 2022-05-11 00:00:00
Tasks: 3D Scene Reconstruction, Autonomous Driving, Autonomous Vehicles, Image-to-Image Translation, Neural Rendering, Novel View Synthesis, Translation

Synthesizing free-view photo-realistic images is an important task in multimedia. With the development of advanced driver assistance systems~(ADAS) and their applications in autonomous vehicles, experimenting with different scenarios becomes a challenge. Although the photo-realistic street scenes can be synthesized by image-to-image translation methods, which cannot produce coherent scenes due to the lack of 3D information. In this paper, a large-scale neural rendering method is proposed to synthesize the autonomous driving scene~(READ), which makes it possible to synthesize large-scale driving scenarios on a PC through a variety of sampling schemes. In order to represent driving scenarios, we propose an {\omega} rendering network to learn neural descriptors from sparse point clouds. Our model can not only synthesize realistic driving scenes but also stitch and edit driving scenes. Experiments show that our model performs well in large-scale driving scenarios.
An Empirical Study Of Self-supervised Learning Approaches For Object Detection With Transformers
Authors: Gokul Karthik Kumar, Sahal Shaji Mullappilly, Abhishek Singh Gehlot
Date: 2022-05-11 00:00:00
Tasks: image-classification, Image Classification, Image Reconstruction, Language Modeling, Language Modelling, Masked Language Modeling, Multi-Task Learning, object-detection, Object Detection, Self-Supervised Learning

Self-supervised learning (SSL) methods such as masked language modeling have shown massive performance gains by pretraining transformer models for a variety of natural language processing tasks. The follow-up research adapted similar methods like masked image modeling in vision transformer and demonstrated improvements in the image classification task. Such simple self-supervised methods are not exhaustively studied for object detection transformers (DETR, Deformable DETR) as their transformer encoder modules take input in the convolutional neural network (CNN) extracted feature space rather than the image space as in general vision transformers. However, the CNN feature maps still maintain the spatial relationship and we utilize this property to design self-supervised learning approaches to train the encoder of object detection transformers in pretraining and multi-task learning settings. We explore common self-supervised methods based on image reconstruction, masked image modeling and jigsaw. Preliminary experiments in the iSAID dataset demonstrate faster convergence of DETR in the initial epochs in both pretraining and multi-task learning settings; nonetheless, similar improvement is not observed in the case of multi-task learning with Deformable DETR. The code for our experiments with DETR and Deformable DETR are available at https://github.com/gokulkarthik/detr and https://github.com/gokulkarthik/Deformable-DETR respectively.
NMR: Neural Manifold Representation for Autonomous Driving
Authors: Unnikrishnan R. Nair, Sarthak Sharma, Midhun S. Menon, Srikanth Vidapanakal
Date: 2022-05-11 00:00:00
Tasks: Autonomous Driving

Autonomous driving requires efficient reasoning about the Spatio-temporal nature of the semantics of the scene. Recent approaches have successfully amalgamated the traditional modular architecture of an autonomous driving stack comprising perception, prediction, and planning in an end-to-end trainable system. Such a system calls for a shared latent space embedding with interpretable intermediate trainable projected representation. One such successfully deployed representation is the Bird's-Eye View(BEV) representation of the scene in ego-frame. However, a fundamental assumption for an undistorted BEV is the local coplanarity of the world around the ego-vehicle. This assumption is highly restrictive, as roads, in general, do have gradients. The resulting distortions make path planning inefficient and incorrect. To overcome this limitation, we propose Neural Manifold Representation (NMR), a representation for the task of autonomous driving that learns to infer semantics and predict way-points on a manifold over a finite horizon, centered on the ego-vehicle. We do this using an iterative attention mechanism applied on a latent high dimensional embedding of surround monocular images and partial ego-vehicle state. This representation helps generate motion and behavior plans consistent with and cognizant of the surface geometry. We propose a sampling algorithm based on edge-adaptive coverage loss of BEV occupancy grid and associated guidance flow field to generate the surface manifold while incurring minimal computational overhead. We aim to test the efficacy of our approach on CARLA and SYNTHIA-SF.