pwc-archive/papers-with-abstracts huggingface dataset:
CE-Dedup: Cost-Effective Convolutional Neural Nets Training based on Image Deduplication
Authors: Xuan Li, Liqiong Chang, Xue Liu
Date: 2021-08-23 00:00:00
Tasks: image-classification, Image Classification

Attributed to the ever-increasing large image datasets, Convolutional Neural Networks (CNNs) have become popular for vision-based tasks. It is generally admirable to have larger-sized datasets for higher network training accuracies. However, the impact of dataset quality has not to be involved. It is reasonable to assume the near-duplicate images exist in the datasets. For instance, the Street View House Numbers (SVHN) dataset having cropped house plate digits from 0 to 9 are likely to have repetitive digits from the same/similar house plates. Redundant images may take up a certain portion of the dataset without consciousness. While contributing little to no accuracy improvement for the CNNs training, these duplicated images unnecessarily pose extra resource and computation consumption. To this end, this paper proposes a framework to assess the impact of the near-duplicate images on CNN training performance, called CE-Dedup. Specifically, CE-Dedup associates a hashing-based image deduplication approach with downstream CNNs-based image classification tasks. CE-Dedup balances the tradeoff between a large deduplication ratio and a stable accuracy by adjusting the deduplication threshold. The effectiveness of CE-Dedup is validated through extensive experiments on well-known CNN benchmarks. On one hand, while maintaining the same validation accuracy, CE-Dedup can reduce the dataset size by 23%. On the other hand, when allowing a small validation accuracy drop (by 5%), CE-Dedup can trim the dataset size by 75%.
Spatio-temporal-spectral-angular observation model that integrates observations from UAV and mobile mapping vehicle for better urban mapping
Authors: Zhenfeng Shao, Gui Cheng, Deren Li, Xiao Huang, Zhipeng Lu, Jian Liu
Date: 2021-08-24 00:00:00

In a complex urban scene, observation from a single sensor unavoidably leads to voids in observations, failing to describe urban objects in a comprehensive manner. In this paper, we propose a spatio-temporal-spectral-angular observation model to integrate observations from UAV and mobile mapping vehicle platform, realizing a joint, coordinated observation operation from both air and ground. We develop a multi-source remote sensing data acquisition system to effectively acquire multi-angle data of complex urban scenes. Multi-source data fusion solves the missing data problem caused by occlusion and achieves accurate, rapid, and complete collection of holographic spatial and temporal information in complex urban scenes. We carried out an experiment on Baisha Town, Chongqing, China and obtained multi-sensor, multi-angle data from UAV and mobile mapping vehicle. We first extracted the point cloud from UAV and then integrated the UAV and mobile mapping vehicle point cloud. The integrated results combined both the characteristic of UAV and mobile mapping vehicle point cloud, confirming the practicability of the proposed joint data acquisition platform and the effectiveness of spatio-temporal-spectral-angular observation model. Compared with the observation from UAV or mobile mapping vehicle alone, the integrated system provides an effective data acquisition solution towards comprehensive urban monitoring.
An Automated Approach for the Recognition of Bengali License Plates
Authors: MD Abdullah Al Nasim, Atiqul Islam Chowdhury, Jannatun Naeem Muna, Faisal Muhammad Shah
Date: 2021-09-01 00:00:00
Tasks: License Plate Detection

Automatic Number Plate Recognition (ALPR) is a system for automatically identifying the license plates of any vehicle. This process is important for tracking, ticketing, and any billing system, among other things. With the use of information and communication technology (ICT), all systems are being automated, including the vehicle tracking system. This study proposes a hybrid method for detecting license plates using characters from them. Our captured image information was used for the recognition procedure in Bangladeshi vehicles, which is the topic of this study. Here, for license plate detection, the YOLO model was used where 81% was correctly predicted. And then, for license plate segmentation, Otsu's Thresholding was used and eventually, for character recognition, the CNN model was applied. This model will allow the vehicle's automated license plate detection system to avoid any misuse.
FA-GAN: Feature-Aware GAN for Text to Image Synthesis
Authors: Eunyeong Jeon, Kunhee Kim, Daijin Kim
Date: 2021-09-02 00:00:00
Tasks: Decoder, Generative Adversarial Network, Image Generation

Text-to-image synthesis aims to generate a photo-realistic image from a given natural language description. Previous works have made significant progress with Generative Adversarial Networks (GANs). Nonetheless, it is still hard to generate intact objects or clear textures (Fig 1). To address this issue, we propose Feature-Aware Generative Adversarial Network (FA-GAN) to synthesize a high-quality image by integrating two techniques: a self-supervised discriminator and a feature-aware loss. First, we design a self-supervised discriminator with an auxiliary decoder so that the discriminator can extract better representation. Secondly, we introduce a feature-aware loss to provide the generator more direct supervision by employing the feature representation from the self-supervised discriminator. Experiments on the MS-COCO dataset show that our proposed method significantly advances the state-of-the-art FID score from 28.92 to 24.58.
Reiterative Domain Aware Multi-Target Adaptation
Authors: Sudipan Saha, Shan Zhao, Nasrullah Sheikh, Xiao Xiang Zhu
Date: 2021-08-26 00:00:00
Tasks: Domain Adaptation, Graph Neural Network, Multi-target Domain Adaptation

Most domain adaptation methods focus on single-source-single-target adaptation settings. Multi-target domain adaptation is a powerful extension in which a single classifier is learned for multiple unlabeled target domains. To build a multi-target classifier, it is important to have: a feature extractor that generalizes well across domains; and effective aggregation of features from the labeled source and different unlabeled target domains. Towards the first, we use the recently popular Transformer as a feature extraction backbone. Towards the second, we use a co-teaching-based approach using a dual-classifier head, one of which is based on the graph neural network. The proposed approach uses a sequential adaptation strategy that adapts one domain at a time starting from the target domains that are more similar to the source, assuming that the network finds it easier to adapt to such target domains. After adapting on each target, samples with a softmax-based confidence score greater than a threshold are added to the pseudo-source, thus aggregating knowledge from different domains. However, softmax is not entirely trustworthy as a confidence score and may generate a high score for unreliable samples if trained for many iterations. To mitigate this effect, we adopt a reiterative approach, where we reduce target adaptation iterations, however, reiterate multiple times over the target domains. The experimental evaluation on the Office-Home, Office-31 and DomainNet datasets shows significant improvement over the existing methods. We have achieved 10.7$\%$ average improvement in Office-Home dataset over the state-of-art methods.
Classifying Organisms and Artefacts By Their Shapes
Authors: Arianna Salili-James, Anne MacKay, Emilio Rodriguez-Alvarez, Diana Rodriguez-Perez, Thomas Mannack, Timothy A. Rawlings, A. Richard Palmer, Jonathan Todd, Terhi E. Riutta, Cate Macinnis-Ng, Zhitong Han, Megan Davies, Zinnia Thorpe, Stephen Marsland, Armand M. Leroi
Date: 2021-08-27 00:00:00

We often wish to classify objects by their shapes. Indeed, the study of shapes is an important part of many scientific fields such as evolutionary biology, structural biology, image processing, and archaeology. The most widely-used method of shape analysis, Geometric Morphometrics, assumes that that the mathematical space in which shapes are represented is linear. However, it has long been known that shape space is, in fact, rather more complicated, and certainly non-linear. Diffeomorphic methods that take this non-linearity into account, and so give more accurate estimates of the distances among shapes, exist but have rarely been applied to real-world problems. Using a machine classifier, we tested the ability of several of these methods to describe and classify the shapes of a variety of organic and man-made objects. We find that one method, the Square-Root Velocity Function (SRVF), is superior to all others, including a standard Geometric Morphometric method (eigenshapes). We also show that computational shape classifiers outperform human experts, and that the SRVF shortest-path between shapes can be used to estimate the shapes of intermediate steps in evolutionary series. Diffeomorphic shape analysis methods, we conclude, now provide practical and effective solutions to many shape description and classification problems in the natural and human sciences.
Impact of Attention on Adversarial Robustness of Image Classification Models
Authors: Prachi Agrawal, Narinder Singh Punn, Sanjay Kumar Sonbhadra, Sonali Agarwal
Date: 2021-09-02 00:00:00
Tasks: Adversarial Robustness, Classification, image-classification, Image Classification

Adversarial attacks against deep learning models have gained significant attention and recent works have proposed explanations for the existence of adversarial examples and techniques to defend the models against these attacks. Attention in computer vision has been used to incorporate focused learning of important features and has led to improved accuracy. Recently, models with attention mechanisms have been proposed to enhance adversarial robustness. Following this context, this work aims at a general understanding of the impact of attention on adversarial robustness. This work presents a comparative study of adversarial robustness of non-attention and attention based image classification models trained on CIFAR-10, CIFAR-100 and Fashion MNIST datasets under the popular white box and black box attacks. The experimental results show that the robustness of attention based models may be dependent on the datasets used i.e. the number of classes involved in the classification. In contrast to the datasets with less number of classes, attention based models are observed to show better robustness towards classification.
SetMargin Loss applied to Deep Keystroke Biometrics with Circle Packing Interpretation
Authors: Aythami Morales, Julian Fierrez, Alejandro Acien, Ruben Tolosana, Ignacio Serna
Date: 2021-09-02 00:00:00
Tasks: Metric Learning

This work presents a new deep learning approach for keystroke biometrics based on a novel Distance Metric Learning method (DML). DML maps input data into a learned representation space that reveals a "semantic" structure based on distances. In this work, we propose a novel DML method specifically designed to address the challenges associated to free-text keystroke identification where the classes used in learning and inference are disjoint. The proposed SetMargin Loss (SM-L) extends traditional DML approaches with a learning process guided by pairs of sets instead of pairs of samples, as done traditionally. The proposed learning strategy allows to enlarge inter-class distances while maintaining the intra-class structure of keystroke dynamics. We analyze the resulting representation space using the mathematical problem known as Circle Packing, which provides neighbourhood structures with a theoretical maximum inter-class distance. We finally prove experimentally the effectiveness of the proposed approach on a challenging task: keystroke biometric identification over a large set of 78,000 subjects. Our method achieves state-of-the-art accuracy on a comparison performed with the best existing approaches.
Extended Object Tracking Using Sets Of Trajectories with a PHD Filter
Authors: Jakob Sjudin, Martin Marcusson, Lennart Svensson, Lars Hammarstrand
Date: 2021-09-02 00:00:00
Tasks: Multiple Object Tracking, Object, Object Tracking

PHD filtering is a common and effective multiple object tracking (MOT) algorithm used in scenarios where the number of objects and their states are unknown. In scenarios where each object can generate multiple measurements per scan, some PHD filters can estimate the extent of the objects as well as their kinematic properties. Most of these approaches are, however, not able to inherently estimate trajectories and rely on ad-hoc methods, such as different labeling schemes, to build trajectories from the state estimates. This paper presents a Gamma Gaussian inverse Wishart mixture PHD filter that can directly estimate sets of trajectories of extended targets by expanding previous research on tracking sets of trajectories for point source objects to handle extended objects. The new filter is compared to an existing extended PHD filter that uses a labeling scheme to build trajectories, and it is shown that the new filter can estimate object trajectories more reliably.
Scene Text recognition with Full Normalization
Authors: Nathan Zachary, Gerald Carl, Russell Elijah, Hessi Roma, Robert Leer, James Amelia
Date: 2021-07-13 00:00:00
Tasks: Optical Character Recognition (OCR), Scene Text Recognition

Scene text recognition has made significant progress in recent years and has become an important part of the work-flow. The widespread use of mobile devices opens up wide possibilities for using OCR technologies in everyday life. However, lack of training data for new research in this area remains relevant. In this article, we present a new dataset consisting of real shots on smartphones and demonstrate the effectiveness of profile normalization in this task. In addition, the influence of various augmentations during the training of models for analyzing document images on smartphones is studied in detail. Our dataset is publicly available.
Shot boundary detection method based on a new extensive dataset and mixed features
Authors: Alexander Gushchin, Anastasia Antsiferova, Dmitriy Vatolin
Date: 2021-09-02 00:00:00
Tasks: Boundary Detection, Change Detection, Scene Change Detection

Shot boundary detection in video is one of the key stages of video data processing. A new method for shot boundary detection based on several video features, such as color histograms and object boundaries, has been proposed. The developed algorithm was tested on the open BBC Planet Earth [1] and RAI [2] datasets, and the MSU CC datasets, based on videos used in the video codec comparison conducted at MSU, as well as videos from the IBM set, were also plotted. The total dataset for algorithm development and testing exceeded the known TRECVID datasets. Based on the test results, the proposed algorithm for scene change detection outperformed its counterparts with a final F-score of 0.9794.
4D-Net for Learned Multi-Modal Alignment
Authors: AJ Piergiovanni, Vincent Casser, Michael S. Ryoo, Anelia Angelova
Date: 2021-09-02 00:00:00
Proceeding: ICCV 2021 10
Tasks: 3D Object Detection, object-detection, Object Detection

We present 4D-Net, a 3D object detection approach, which utilizes 3D Point Cloud and RGB sensing information, both in time. We are able to incorporate the 4D information by performing a novel dynamic connection learning across various feature representations and levels of abstraction, as well as by observing geometric constraints. Our approach outperforms the state-of-the-art and strong baselines on the Waymo Open Dataset. 4D-Net is better able to use motion cues and dense image information to detect distant objects more successfully.
SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpainting
Authors: Varun Jampani, Huiwen Chang, Kyle Sargent, Abhishek Kar, Richard Tucker, Michael Krainin, Dominik Kaeser, William T. Freeman, David Salesin, Brian Curless, Ce Liu
Date: 2021-09-02 00:00:00
Proceeding: ICCV 2021 10
Tasks: Image Matting

Single image 3D photography enables viewers to view a still image from novel viewpoints. Recent approaches combine monocular depth networks with inpainting networks to achieve compelling results. A drawback of these techniques is the use of hard depth layering, making them unable to model intricate appearance details such as thin hair-like structures. We present SLIDE, a modular and unified system for single image 3D photography that uses a simple yet effective soft layering strategy to better preserve appearance details in novel views. In addition, we propose a novel depth-aware training strategy for our inpainting module, better suited for the 3D photography task. The resulting SLIDE approach is modular, enabling the use of other components such as segmentation and matting for improved layering. At the same time, SLIDE uses an efficient layered depth formulation that only requires a single forward pass through the component networks to produce high quality 3D photos. Extensive experimental analysis on three view-synthesis datasets, in combination with user studies on in-the-wild image collections, demonstrate superior performance of our technique in comparison to existing strong baselines while being conceptually much simpler. Project page: https://varunjampani.github.io/slide
Domain-Robust Mitotic Figure Detection with Style Transfer
Authors: Youjin Chung, Jihoon Cho, Jinah Park
Date: 2021-09-02 00:00:00
Tasks: Domain Generalization, Style Transfer

We propose a new training scheme for domain generalization in mitotic figure detection. Mitotic figures show different characteristics for each scanner. We consider each scanner as a 'domain' and the image distribution specified for each domain as 'style'. The goal is to train our network to be robust on scanner types by using various 'style' images. To expand the style variance, we transfer a style of the training image into arbitrary styles, by defining a module based on StarGAN. Our model with the proposed training scheme shows positive performance on MIDOG Preliminary Test-Set containing scanners never seen before.
NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo
Authors: Yi Wei, Shaohui Liu, Yongming Rao, Wang Zhao, Jiwen Lu, Jie zhou
Date: 2021-09-02 00:00:00
Proceeding: ICCV 2021 10
Tasks: Depth Estimation, NeRF

In this work, we present a new multi-view depth estimation method that utilizes both conventional reconstruction and learning-based priors over the recently proposed neural radiance fields (NeRF). Unlike existing neural network based optimization method that relies on estimated correspondences, our method directly optimizes over implicit volumes, eliminating the challenging step of matching pixels in indoor scenes. The key to our approach is to utilize the learning-based priors to guide the optimization process of NeRF. Our system firstly adapts a monocular depth network over the target scene by finetuning on its sparse SfM+MVS reconstruction from COLMAP. Then, we show that the shape-radiance ambiguity of NeRF still exists in indoor environments and propose to address the issue by employing the adapted depth priors to monitor the sampling process of volume rendering. Finally, a per-pixel confidence map acquired by error computation on the rendered image can be used to further improve the depth quality. Experiments show that our proposed framework significantly outperforms state-of-the-art methods on indoor scenes, with surprising findings presented on the effectiveness of correspondence-based optimization and NeRF-based optimization over the adapted depth priors. In addition, we show that the guided optimization scheme does not sacrifice the original synthesis capability of neural radiance fields, improving the rendering quality on both seen and novel views. Code is available at https://github.com/weiyithu/NerfingMVS.
A New Semi-Automated Algorithm for Volumetric Segmentation of the Left Ventricle in Temporal 3D Echocardiography Sequences
Authors: Deepa Krishnaswamy, Abhilash R. Hareendranathan, Tan Suwatanaviroj, Pierre Boulanger, Harald Becher, Michelle Noga, Kumaradevan Punithakumar
Date: 2021-09-02 00:00:00

Purpose: Echocardiography is commonly used as a non-invasive imaging tool in clinical practice for the assessment of cardiac function. However, delineation of the left ventricle is challenging due to the inherent properties of ultrasound imaging, such as the presence of speckle noise and the low signal-to-noise ratio. Methods: We propose a semi-automated segmentation algorithm for the delineation of the left ventricle in temporal 3D echocardiography sequences. The method requires minimal user interaction and relies on a diffeomorphic registration approach. Advantages of the method include no dependence on prior geometrical information, training data, or registration from an atlas. Results: The method was evaluated using three-dimensional ultrasound scan sequences from 18 patients from the Mazankowski Alberta Heart Institute, Edmonton, Canada, and compared to manual delineations provided by an expert cardiologist and four other registration algorithms. The segmentation approach yielded the following results over the cardiac cycle: a mean absolute difference of 1.01 (0.21) mm, a Hausdorff distance of 4.41 (1.43) mm, and a Dice overlap score of 0.93 (0.02). Conclusions: The method performed well compared to the four other registration algorithms.
The Power of Points for Modeling Humans in Clothing
Authors: Qianli Ma, Jinlong Yang, Siyu Tang, Michael J. Black
Date: 2021-09-02 00:00:00
Proceeding: ICCV 2021 10

Currently it requires an artist to create 3D human avatars with realistic clothing that can move naturally. Despite progress on 3D scanning and modeling of human bodies, there is still no technology that can easily turn a static scan into an animatable avatar. Automating the creation of such avatars would enable many applications in games, social networking, animation, and AR/VR to name a few. The key problem is one of representation. Standard 3D meshes are widely used in modeling the minimally-clothed body but do not readily capture the complex topology of clothing. Recent interest has shifted to implicit surface models for this task but they are computationally heavy and lack compatibility with existing 3D tools. What is needed is a 3D representation that can capture varied topology at high resolution and that can be learned from data. We argue that this representation has been with us all along -- the point cloud. Point clouds have properties of both implicit and explicit representations that we exploit to model 3D garment geometry on a human body. We train a neural network with a novel local clothing geometric feature to represent the shape of different outfits. The network is trained from 3D point clouds of many types of clothing, on many bodies, in many poses, and learns to model pose-dependent clothing deformations. The geometry feature can be optimized to fit a previously unseen scan of a person in clothing, enabling the scan to be reposed realistically. Our model demonstrates superior quantitative and qualitative results in both multi-outfit modeling and unseen outfit animation. The code is available for research purposes.