pwc-archive/papers-with-abstracts huggingface dataset:
Swin Transformer coupling CNNs Makes Strong Contextual Encoders for VHR Image Road Extraction
Authors: Tao Chen, Yiran Liu, Haoyu Jiang, Ruirui Li
Date: 2022-01-10 00:00:00
Tasks: Decoder

Accurately segmenting roads is challenging due to substantial intra-class variations, indistinct inter-class distinctions, and occlusions caused by shadows, trees, and buildings. To address these challenges, attention to important texture details and perception of global geometric contextual information are essential. Recent research has shown that CNN-Transformer hybrid structures outperform using CNN or Transformer alone. While CNN excels at extracting local detail features, the Transformer naturally perceives global contextual information. In this paper, we propose a dual-branch network block named ConSwin that combines ResNet and SwinTransformers for road extraction tasks. This ConSwin block harnesses the strengths of both approaches to better extract detailed and global features. Based on ConSwin, we construct an hourglass-shaped road extraction network and introduce two novel connection structures to better transmit texture and structural detail information to the decoder. Our proposed method outperforms state-of-the-art methods on both the Massachusetts and CHN6-CUG datasets in terms of overall accuracy, IOU, and F1 indicators. Additional experiments validate the effectiveness of our proposed module, while visualization results demonstrate its ability to obtain better road representations.
Pedestrian Detection: Domain Generalization, CNNs, Transformers and Beyond
Authors: Irtiza Hasan, Shengcai Liao, Jinpeng Li, Saad Ullah Akram, Ling Shao
Date: 2022-01-10 00:00:00
Tasks: Attribute, Autonomous Driving, Domain Generalization, object-detection, Object Detection, Object Tracking, Pedestrian Detection

Pedestrian detection is the cornerstone of many vision based applications, starting from object tracking to video surveillance and more recently, autonomous driving. With the rapid development of deep learning in object detection, pedestrian detection has achieved very good performance in traditional single-dataset training and evaluation setting. However, in this study on generalizable pedestrian detectors, we show that, current pedestrian detectors poorly handle even small domain shifts in cross-dataset evaluation. We attribute the limited generalization to two main factors, the method and the current sources of data. Regarding the method, we illustrate that biasness present in the design choices (e.g anchor settings) of current pedestrian detectors are the main contributing factor to the limited generalization. Most modern pedestrian detectors are tailored towards target dataset, where they do achieve high performance in traditional single training and testing pipeline, but suffer a degrade in performance when evaluated through cross-dataset evaluation. Consequently, a general object detector performs better in cross-dataset evaluation compared with state of the art pedestrian detectors, due to its generic design. As for the data, we show that the autonomous driving benchmarks are monotonous in nature, that is, they are not diverse in scenarios and dense in pedestrians. Therefore, benchmarks curated by crawling the web (which contain diverse and dense scenarios), are an efficient source of pre-training for providing a more robust representation. Accordingly, we propose a progressive fine-tuning strategy which improves generalization. Code and models can accessed at https://github.com/hasanirtiza/Pedestron.
TFS Recognition: Investigating MPH]{Thai Finger Spelling Recognition: Investigating MediaPipe Hands Potentials
Authors: Jinnavat Sanalohit, Tatpong Katanyukul
Date: 2022-01-10 00:00:00
Tasks: Keypoint Detection

Thai Finger Spelling (TFS) sign recognition could benefit a community of hearing-difficulty people in bridging to a major hearing population. With a relatively large number of alphabets, TFS employs multiple signing schemes. Two schemes of more common signing -- static and dynamic single-hand signing, widely used in other sign languages -- have been addressed in several previous works. To complete the TFS sign recognition, the remaining two of quite distinct signing schemes -- static and dynamic point-on-hand signing -- need to be sufficiently addressed. With the advent of many off-the-shelf hand skeleton prediction models and that training a model to recognize a sign language from scratch is expensive, we explore an approach building upon recently launched MediaPipe Hands (MPH). MPH is a high-precision well-trained model for hand-keypoint detection. We have investigated MPH on three TFS schemes: static-single-hand (S1), simplified dynamic-single-hand (S2) and static-point-on-hand (P1) schemes. Our results show that MPH can satisfactorily address single-hand schemes with accuracy of 84.57% on both S1 and S2. However, our finding reveals a shortcoming of MPH in addressing a point-on-hand scheme, whose accuracy is 23.66% on P1 conferring to 69.19% obtained from conventional classification trained from scratch. This shortcoming has been investigated and attributed to self occlusion and handedness.
Enhancing Low-Light Images in Real World via Cross-Image Disentanglement
Authors: Lanqing Guo, Renjie Wan, Wenhan Yang, Alex Kot, Bihan Wen
Date: 2022-01-10 00:00:00
Tasks: Disentanglement, Image Enhancement, Low-Light Image Enhancement

Images captured in the low-light condition suffer from low visibility and various imaging artifacts, e.g., real noise. Existing supervised enlightening algorithms require a large set of pixel-aligned training image pairs, which are hard to prepare in practice. Though weakly-supervised or unsupervised methods can alleviate such challenges without using paired training images, some real-world artifacts inevitably get falsely amplified because of the lack of corresponded supervision. In this paper, instead of using perfectly aligned images for training, we creatively employ the misaligned real-world images as the guidance, which are considerably easier to collect. Specifically, we propose a Cross-Image Disentanglement Network (CIDN) to separately extract cross-image brightness and image-specific content features from low/normal-light images. Based on that, CIDN can simultaneously correct the brightness and suppress image artifacts in the feature domain, which largely increases the robustness to the pixel shifts. Furthermore, we collect a new low-light image enhancement dataset consisting of misaligned training images with real-world corruptions. Experimental results show that our model achieves state-of-the-art performances on both the newly proposed dataset and other popular low-light datasets.
Multi-Level Attention for Unsupervised Person Re-Identification
Authors: Yi Zheng
Date: 2022-01-10 00:00:00
Tasks: Person Re-Identification, Unsupervised Person Re-Identification

The attention mechanism is widely used in deep learning because of its excellent performance in neural networks without introducing additional information. However, in unsupervised person re-identification, the attention module represented by multi-headed self-attention suffers from attention spreading in the condition of non-ground truth. To solve this problem, we design pixel-level attention module to provide constraints for multi-headed self-attention. Meanwhile, for the trait that the identification targets of person re-identification data are all pedestrians in the samples, we design domain-level attention module to provide more comprehensive pedestrian features. We combine head-level, pixel-level and domain-level attention to propose multi-level attention block and validate its performance on for large person re-identification datasets (Market-1501, DukeMTMC-reID and MSMT17 and PersonX).
Systematic biases when using deep neural networks for annotating large catalogs of astronomical images
Authors: Sanchari Dhar, Lior Shamir
Date: 2022-01-10 00:00:00

Deep convolutional neural networks (DCNNs) have become the most common solution for automatic image annotation due to their non-parametric nature, good performance, and their accessibility through libraries such as TensorFlow. Among other fields, DCNNs are also a common approach to the annotation of large astronomical image databases acquired by digital sky surveys. One of the main downsides of DCNNs is the complex non-intuitive rules that make DCNNs act as a ``black box", providing annotations in a manner that is unclear to the user. Therefore, the user is often not able to know what information is used by the DCNNs for the classification. Here we demonstrate that the training of a DCNN is sensitive to the context of the training data such as the location of the objects in the sky. We show that for basic classification of elliptical and spiral galaxies, the sky location of the galaxies used for training affects the behavior of the algorithm, and leads to a small but consistent and statistically significant bias. That bias exhibits itself in the form of cosmological-scale anisotropy in the distribution of basic galaxy morphology. Therefore, while DCNNs are powerful tools for annotating images of extended sources, the construction of training sets for galaxy morphology should take into consideration more aspects than the visual appearance of the object. In any case, catalogs created with deep neural networks that exhibit signs of cosmological anisotropy should be interpreted with the possibility of consistent bias.
Signal Reconstruction from Quantized Noisy Samples of the Discrete Fourier Transform
Authors: Mohak Goyal, Animesh Kumar
Date: 2022-01-09 00:00:00
Tasks: Image Reconstruction, MS-SSIM, SSIM

In this paper, we present two variations of an algorithm for signal reconstruction from one-bit or two-bit noisy observations of the discrete Fourier transform (DFT). The one-bit observations of the DFT correspond to the sign of its real part, whereas, the two-bit observations of the DFT correspond to the signs of both the real and imaginary parts of the DFT. We focus on images for analysis and simulations, thus using the sign of the 2D-DFT. This choice of the class of signals is inspired by previous works on this problem. For our algorithm, we show that the expected mean squared error (MSE) in signal reconstruction is asymptotically proportional to the inverse of the sampling rate. The samples are affected by additive zero-mean noise of known distribution. We solve this signal estimation problem by designing an algorithm that uses contraction mapping, based on the Banach fixed point theorem. Numerical tests with four benchmark images are provided to show the effectiveness of our algorithm. Various metrics for image reconstruction quality assessment such as PSNR, SSIM, ESSIM, and MS-SSIM are employed. On all four benchmark images, our algorithm outperforms the state-of-the-art in all of these metrics by a significant margin.
Preserving Domain Private Representation via Mutual Information Maximization
Authors: Jiahong Chen, Jing Wang, Weipeng Lin, Kuangen Zhang, Clarence W. de Silva
Date: 2022-01-09 00:00:00
Tasks: Domain Adaptation, Domain Generalization, Unsupervised Domain Adaptation

Recent advances in unsupervised domain adaptation have shown that mitigating the domain divergence by extracting the domain-invariant representation could significantly improve the generalization of a model to an unlabeled data domain. Nevertheless, the existing methods fail to effectively preserve the representation that is private to the label-missing domain, which could adversely affect the generalization. In this paper, we propose an approach to preserve such representation so that the latent distribution of the unlabeled domain could represent both the domain-invariant features and the individual characteristics that are private to the unlabeled domain. In particular, we demonstrate that maximizing the mutual information between the unlabeled domain and its latent space while mitigating the domain divergence can achieve such preservation. We also theoretically and empirically validate that preserving the representation that is private to the unlabeled domain is important and of necessity for the cross-domain generalization. Our approach outperforms state-of-the-art methods on several public datasets.
ImageSubject: A Large-scale Dataset for Subject Detection
Authors: Xin Miao, Jiayi Liu, Huayan Wang, Jun Fu
Date: 2022-01-09 00:00:00
Tasks: object-detection, Object Detection, Saliency Detection

Main subjects usually exist in the images or videos, as they are the objects that the photographer wants to highlight. Human viewers can easily identify them but algorithms often confuse them with other objects. Detecting the main subjects is an important technique to help machines understand the content of images and videos. We present a new dataset with the goal of training models to understand the layout of the objects and the context of the image then to find the main subjects among them. This is achieved in three aspects. By gathering images from movie shots created by directors with professional shooting skills, we collect the dataset with strong diversity, specifically, it contains 107\,700 images from 21\,540 movie shots. We labeled them with the bounding box labels for two classes: subject and non-subject foreground object. We present a detailed analysis of the dataset and compare the task with saliency detection and object detection. ImageSubject is the first dataset that tries to localize the subject in an image that the photographer wants to highlight. Moreover, we find the transformer-based detection model offers the best result among other popular model architectures. Finally, we discuss the potential applications and conclude with the importance of the dataset.
COVID-19 Infection Segmentation from Chest CT Images Based on Scale Uncertainty
Authors: Masahiro Oda, Tong Zheng, Yuichiro Hayashi, Yoshito Otake, Masahiro Hashimoto, Toshiaki Akashi, Shigeki Aoki, Kensaku MORI
Date: 2022-01-09 00:00:00
Tasks: Prediction, Segmentation

This paper proposes a segmentation method of infection regions in the lung from CT volumes of COVID-19 patients. COVID-19 spread worldwide, causing many infected patients and deaths. CT image-based diagnosis of COVID-19 can provide quick and accurate diagnosis results. An automated segmentation method of infection regions in the lung provides a quantitative criterion for diagnosis. Previous methods employ whole 2D image or 3D volume-based processes. Infection regions have a considerable variation in their sizes. Such processes easily miss small infection regions. Patch-based process is effective for segmenting small targets. However, selecting the appropriate patch size is difficult in infection region segmentation. We utilize the scale uncertainty among various receptive field sizes of a segmentation FCN to obtain infection regions. The receptive field sizes can be defined as the patch size and the resolution of volumes where patches are clipped from. This paper proposes an infection segmentation network (ISNet) that performs patch-based segmentation and a scale uncertainty-aware prediction aggregation method that refines the segmentation result. We design ISNet to segment infection regions that have various intensity values. ISNet has multiple encoding paths to process patch volumes normalized by multiple intensity ranges. We collect prediction results generated by ISNets having various receptive field sizes. Scale uncertainty among the prediction results is extracted by the prediction aggregation method. We use an aggregation FCN to generate a refined segmentation result considering scale uncertainty among the predictions. In our experiments using 199 chest CT volumes of COVID-19 cases, the prediction aggregation method improved the dice similarity score from 47.6% to 62.1%.
Lung infection and normal region segmentation from CT volumes of COVID-19 cases
Authors: Masahiro Oda, Yuichiro Hayashi, Yoshito Otake, Masahiro Hashimoto, Toshiaki Akashi, Kensaku MORI
Date: 2022-01-09 00:00:00
Tasks: Segmentation

This paper proposes an automated segmentation method of infection and normal regions in the lung from CT volumes of COVID-19 patients. From December 2019, novel coronavirus disease 2019 (COVID-19) spreads over the world and giving significant impacts to our economic activities and daily lives. To diagnose the large number of infected patients, diagnosis assistance by computers is needed. Chest CT is effective for diagnosis of viral pneumonia including COVID-19. A quantitative analysis method of condition of the lung from CT volumes by computers is required for diagnosis assistance of COVID-19. This paper proposes an automated segmentation method of infection and normal regions in the lung from CT volumes using a COVID-19 segmentation fully convolutional network (FCN). In diagnosis of lung diseases including COVID-19, analysis of conditions of normal and infection regions in the lung is important. Our method recognizes and segments lung normal and infection regions in CT volumes. To segment infection regions that have various shapes and sizes, we introduced dense pooling connections and dilated convolutions in our FCN. We applied the proposed method to CT volumes of COVID-19 cases. From mild to severe cases of COVID-19, the proposed method correctly segmented normal and infection regions in the lung. Dice scores of normal and infection regions were 0.911 and 0.753, respectively.
Applying Artificial Intelligence for Age Estimation in Digital Forensic Investigations
Authors: Thomas Grubl, Harjinder Singh Lallie
Date: 2022-01-09 00:00:00
Tasks: Age Estimation, Diversity

The precise age estimation of child sexual abuse and exploitation (CSAE) victims is one of the most significant digital forensic challenges. Investigators often need to determine the age of victims by looking at images and interpreting the sexual development stages and other human characteristics. The main priority - safeguarding children -- is often negatively impacted by a huge forensic backlog, cognitive bias and the immense psychological stress that this work can entail. This paper evaluates existing facial image datasets and proposes a new dataset tailored to the needs of similar digital forensic research contributions. This small, diverse dataset of 0 to 20-year-old individuals contains 245 images and is merged with 82 unique images from the FG-NET dataset, thus achieving a total of 327 images with high image diversity and low age range density. The new dataset is tested on the Deep EXpectation (DEX) algorithm pre-trained on the IMDB-WIKI dataset. The overall results for young adolescents aged 10 to 15 and older adolescents/adults aged 16 to 20 are very encouraging -- achieving MAEs as low as 1.79, but also suggest that the accuracy for children aged 0 to 10 needs further work. In order to determine the efficacy of the prototype, valuable input of four digital forensic experts, including two forensic investigators, has been taken into account to improve age estimation results. Further research is required to extend datasets both concerning image density and the equal distribution of factors such as gender and racial diversity.
Semantics-driven Attentive Few-shot Learning over Clean and Noisy Samples
Authors: Orhun Buğra Baran, Ramazan Gökberk Cinbiş
Date: 2022-01-09 00:00:00
Tasks: Few-Shot Learning

Over the last couple of years few-shot learning (FSL) has attracted great attention towards minimizing the dependency on labeled training examples. An inherent difficulty in FSL is the handling of ambiguities resulting from having too few training samples per class. To tackle this fundamental challenge in FSL, we aim to train meta-learner models that can leverage prior semantic knowledge about novel classes to guide the classifier synthesis process. In particular, we propose semantically-conditioned feature attention and sample attention mechanisms that estimate the importance of representation dimensions and training instances. We also study the problem of sample noise in FSL, towards the utilization of meta-learners in more realistic and imperfect settings. Our experimental results demonstrate the effectiveness of the proposed semantic FSL model with and without sample noise.
Self-Supervised Feature Learning from Partial Point Clouds via Pose Disentanglement
Authors: Meng-Shiun Tsai, Pei-Ze Chiang, Yi-Hsuan Tsai, Wei-Chen Chiu
Date: 2022-01-09 00:00:00
Tasks: Disentanglement, Representation Learning, Self-Supervised Learning

Self-supervised learning on point clouds has gained a lot of attention recently, since it addresses the label-efficiency and domain-gap problems on point cloud tasks. In this paper, we propose a novel self-supervised framework to learn informative representations from partial point clouds. We leverage partial point clouds scanned by LiDAR that contain both content and pose attributes, and we show that disentangling such two factors from partial point clouds enhances feature representation learning. To this end, our framework consists of three main parts: 1) a completion network to capture holistic semantics of point clouds; 2) a pose regression network to understand the viewing angle where partial data is scanned from; 3) a partial reconstruction network to encourage the model to learn content and pose features. To demonstrate the robustness of the learnt feature representations, we conduct several downstream tasks including classification, part segmentation, and registration, with comparisons against state-of-the-art methods. Our method not only outperforms existing self-supervised methods, but also shows a better generalizability across synthetic and real-world datasets.
Learning from Synthetic InSAR with Vision Transformers: The case of volcanic unrest detection
Authors: Nikolaos Ioannis Bountos, Dimitrios Michail, Ioannis Papoutsis
Date: 2022-01-09 00:00:00
Tasks: Binary Classification, Domain Adaptation

The detection of early signs of volcanic unrest preceding an eruption, in the form of ground deformation in Interferometric Synthetic Aperture Radar (InSAR) data is critical for assessing volcanic hazard. In this work we treat this as a binary classification problem of InSAR images, and propose a novel deep learning methodology that exploits a rich source of synthetically generated interferograms to train quality classifiers that perform equally well in real interferograms. The imbalanced nature of the problem, with orders of magnitude fewer positive samples, coupled with the lack of a curated database with labeled InSAR data, sets a challenging task for conventional deep learning architectures. We propose a new framework for domain adaptation, in which we learn class prototypes from synthetic data with vision transformers. We report detection accuracy that amounts to the highest reported accuracy on a large test set for volcanic unrest detection. Moreover, we built upon this knowledge by learning a new, non-linear, projection between the learnt representations and prototype space, using pseudo labels produced by our model from an unlabeled real InSAR dataset. This leads to the new state of the art with 97.1% accuracy on our test set. We demonstrate the robustness of our approach by training a simple ResNet-18 Convolutional Neural Network on the unlabeled real InSAR dataset with pseudo-labels generated from our top transformer-prototype model. Our methodology provides a significant improvement in performance without the need of manually labeling any sample, opening the road for further exploitation of synthetic InSAR data in various remote sensing applications.