pwc-archive/papers-with-abstracts huggingface dataset:
Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection
Authors: Yuxiang Zhang, Sachin Mehta, Anat Caspi
Date: 2021-01-21 00:00:00
Tasks: Autonomous Navigation, Model Selection, Scene Understanding, Segmentation, Semantic Segmentation

Semantic segmentation aims to robustly predict coherent class labels for entire regions of an image. It is a scene understanding task that powers real-world applications (e.g., autonomous navigation). One important application, the use of imagery for automated semantic understanding of pedestrian environments, provides remote mapping of accessibility features in street environments. This application (and others like it) require detailed geometric information of geographical objects. Semantic segmentation is a prerequisite for this task since it maps contiguous regions of the same class as single entities. Importantly, semantic segmentation uses like ours are not pixel-wise outcomes; however, most of their quantitative evaluation metrics (e.g., mean Intersection Over Union) are based on pixel-wise similarities to a ground-truth, which fails to emphasize over- and under-segmentation properties of a segmentation model. Here, we introduce a new metric to assess region-based over- and under-segmentation. We analyze and compare it to other metrics, demonstrating that the use of our metric lends greater explainability to semantic segmentation model performance in real-world applications.
Video Summarization: Study of various techniques
Authors: Ravi Raj, Varad Bhatnagar, Aman Kumar Singh, Sneha Mane, Nilima Walde
Date: 2021-01-21 00:00:00
Tasks: All, Video Summarization

A comparative study of various techniques which can be used for summarization of Videos i.e. Video to Video conversion is presented along with respective architecture, results, strengths and shortcomings. In all approaches, a lengthy video is converted into a shorter video which aims to capture all important events that are present in the original video. The definition of 'important event' may vary according to the context, such as a sports video and a documentary may have different events which are classified as important.
All-Day Object Tracking for Unmanned Aerial Vehicle
Authors: Bowen Li, Changhong Fu, Fangqiang Ding, Junjie Ye, Fuling Lin
Date: 2021-01-21 00:00:00
Tasks: All, CPU, Object, Object Tracking, Visual Object Tracking

Visual object tracking, which is representing a major interest in image processing field, has facilitated numerous real world applications. Among them, equipping unmanned aerial vehicle (UAV) with real time robust visual trackers for all day aerial maneuver, is currently attracting incremental attention and has remarkably broadened the scope of applications of object tracking. However, prior tracking methods have merely focused on robust tracking in the well-illuminated scenes, while ignoring trackers' capabilities to be deployed in the dark. In darkness, the conditions can be more complex and harsh, easily posing inferior robust tracking or even tracking failure. To this end, this work proposed a novel discriminative correlation filter based tracker with illumination adaptive and anti dark capability, namely ADTrack. ADTrack firstly exploits image illuminance information to enable adaptability of the model to the given light condition. Then, by virtue of an efficient and effective image enhancer, ADTrack carries out image pretreatment, where a target aware mask is generated. Benefiting from the mask, ADTrack aims to solve a dual regression problem where dual filters, i.e., the context filter and target focused filter, are trained with mutual constraint. Thus ADTrack is able to maintain continuously favorable performance in all-day conditions. Besides, this work also constructed one UAV nighttime tracking benchmark UAVDark135, comprising of more than 125k manually annotated frames, which is also very first UAV nighttime tracking benchmark. Exhaustive experiments are extended on authoritative daytime benchmarks, i.e., UAV123 10fps, DTB70, and the newly built dark benchmark UAVDark135, which have validated the superiority of ADTrack in both bright and dark conditions on a single CPU.
Fire Threat Detection From Videos with Q-Rough Sets
Authors: Debarati B. Chakrabortya, Vinay Detania, Shah Parshv Jigneshkumar
Date: 2021-01-21 00:00:00
Tasks: Q-Learning, Segmentation

This article defines new methods for unsupervised fire region segmentation and fire threat detection from video stream. Fire in control serves a number of purposes to human civilization, but it could simultaneously be a threat once its spread becomes uncontrolled. There exists many methods on fire region segmentation and fire non-fire classification. But the approaches to determine the threat associated with fire is relatively scare, and no such unsupervised method has been formulated yet. Here we focus on developing an unsupervised method with which the threat of fire can be quantified and accordingly generate an alarm in automated surveillance systems in indoor as well as in outdoors. Fire region segmentation without any manual intervention/ labelled data set is a major challenge while formulating such a method. Here we have used rough approximations to approximate the fire region, and to manage the incompleteness of the knowledge base, due to absence of any prior information. Utility maximization of Q-learning has been used to minimize ambiguities in the rough approximations. The new set approximation method, thus developed here, is named as Q-rough set. It is used for fire region segmentation from video frames. The threat index of fire flame over the input video stream has been defined in sync with the relative growth in the fire segments on the recent frames. All theories and indices defined here have been experimentally validated with different types of fire videos, through demonstrations and comparisons, as superior to the state of the art.
Segmenting Transparent Object in the Wild with Transformer
Authors: Enze Xie, Wenjia Wang, Wenhai Wang, Peize Sun, Hang Xu, Ding Liang, Ping Luo
Date: 2021-01-21 00:00:00
Tasks: Decoder, Object, Segmentation, Semantic Segmentation, Transparent objects

This work presents a new fine-grained transparent object segmentation dataset, termed Trans10K-v2, extending Trans10K-v1, the first large-scale transparent object segmentation dataset. Unlike Trans10K-v1 that only has two limited categories, our new dataset has several appealing benefits. (1) It has 11 fine-grained categories of transparent objects, commonly occurring in the human domestic environment, making it more practical for real-world application. (2) Trans10K-v2 brings more challenges for the current advanced segmentation methods than its former version. Furthermore, a novel transformer-based segmentation pipeline termed Trans2Seg is proposed. Firstly, the transformer encoder of Trans2Seg provides the global receptive field in contrast to CNN's local receptive field, which shows excellent advantages over pure CNN architectures. Secondly, by formulating semantic segmentation as a problem of dictionary look-up, we design a set of learnable prototypes as the query of Trans2Seg's transformer decoder, where each prototype learns the statistics of one category in the whole dataset. We benchmark more than 20 recent semantic segmentation methods, demonstrating that Trans2Seg significantly outperforms all the CNN-based methods, showing the proposed algorithm's potential ability to solve transparent object segmentation.
COLLIDE-PRED: Prediction of On-Road Collision From Surveillance Videos
Authors: Deesha Chavan, Dev Saad, Debarati B. Chakraborty
Date: 2021-01-21 00:00:00
Tasks: Object, object-detection, Object Detection, Object Tracking, Prediction, Trajectory Prediction

Predicting on-road abnormalities such as road accidents or traffic violations is a challenging task in traffic surveillance. If such predictions can be done in advance, many damages can be controlled. Here in our wok, we tried to formulate a solution for automated collision prediction in traffic surveillance videos with computer vision and deep networks. It involves object detection, tracking, trajectory estimation, and collision prediction. We propose an end-to-end collision prediction system, named as COLLIDE-PRED, that intelligently integrates the information of past and future trajectories of moving objects to predict collisions in videos. It is a pipeline that starts with object detection, which is used for object tracking, and then trajectory prediction is performed which concludes by collision detection. The probable place of collision, and the objects those may cause the collision, both can be identified correctly with COLLIDE-PRED. The proposed method is experimentally validated with a number of different videos and proves to be effective in identifying accident in advance.
FWB-Net:Front White Balance Network for Color Shift Correction in Single Image Dehazing via Atmospheric Light Estimation
Authors: Cong Wang, Yan Huang, Yuexian Zou, Yong Xu
Date: 2021-01-21 00:00:00
Tasks: Image Dehazing, Single Image Dehazing

In recent years, single image dehazing deep models based on Atmospheric Scattering Model (ASM) have achieved remarkable results. But the dehazing outputs of those models suffer from color shift. Analyzing the ASM model shows that the atmospheric light factor (ALF) is set as a scalar which indicates ALF is constant for whole image. However, for images taken in real-world, the illumination is not uniformly distributed over whole image which brings model mismatch and possibly results in color shift of the deep models using ASM. Bearing this in mind, in this study, first, a new non-homogeneous atmospheric scattering model (NH-ASM) is proposed for improving image modeling of hazy images taken under complex illumination conditions. Second, a new U-Net based front white balance module (FWB-Module) is dedicatedly designed to correct color shift before generating dehazing result via atmospheric light estimation. Third, a new FWB loss is innovatively developed for training FWB-Module, which imposes penalty on color shift. In the end, based on NH-ASM and front white balance technology, an end-to-end CNN-based color-shift-restraining dehazing network is developed, termed as FWB-Net. Experimental results demonstrate the effectiveness and superiority of our proposed FWB-Net for dehazing on both synthetic and real-world images.
Anti-UAV: A Large Multi-Modal Benchmark for UAV Tracking
Authors: Nan Jiang, Kuiran Wang, Xiaoke Peng, Xuehui Yu, Qiang Wang, Junliang Xing, Guorong Li, Jian Zhao, Guodong Guo, Zhenjun Han
Date: 2021-01-21 00:00:00

Unmanned Aerial Vehicle (UAV) offers lots of applications in both commerce and recreation. With this, monitoring the operation status of UAVs is crucially important. In this work, we consider the task of tracking UAVs, providing rich information such as location and trajectory. To facilitate research on this topic, we propose a dataset, Anti-UAV, with more than 300 video pairs containing over 580k manually annotated bounding boxes. The releasing of such a large-scale dataset could be a useful initial step in research of tracking UAVs. Furthermore, the advancement of addressing research challenges in Anti-UAV can help the design of anti-UAV systems, leading to better surveillance of UAVs. Besides, a novel approach named dual-flow semantic consistency (DFSC) is proposed for UAV tracking. Modulated by the semantic flow across video sequences, the tracker learns more robust class-level semantic information and obtains more discriminative instance-level features. Experimental results demonstrate that Anti-UAV is very challenging, and the proposed method can effectively improve the tracker's performance. The Anti-UAV benchmark and the code of the proposed approach will be publicly available at https://github.com/ucas-vg/Anti-UAV.
CM-NAS: Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identification
Authors: Chaoyou Fu, Yibo Hu, Xiang Wu, Hailin Shi, Tao Mei, Ran He
Date: 2021-01-21 00:00:00
Proceeding: ICCV 2021 10
Tasks: Neural Architecture Search, Person Re-Identification

Visible-Infrared person re-identification (VI-ReID) aims to match cross-modality pedestrian images, breaking through the limitation of single-modality person ReID in dark environment. In order to mitigate the impact of large modality discrepancy, existing works manually design various two-stream architectures to separately learn modality-specific and modality-sharable representations. Such a manual design routine, however, highly depends on massive experiments and empirical practice, which is time consuming and labor intensive. In this paper, we systematically study the manually designed architectures, and identify that appropriately separating Batch Normalization (BN) layers is the key to bring a great boost towards cross-modality matching. Based on this observation, the essential objective is to find the optimal separation scheme for each BN layer. To this end, we propose a novel method, named Cross-Modality Neural Architecture Search (CM-NAS). It consists of a BN-oriented search space in which the standard optimization can be fulfilled subject to the cross-modality task. Equipped with the searched architecture, our method outperforms state-of-the-art counterparts in both two benchmarks, improving the Rank-1/mAP by 6.70%/6.13% on SYSU-MM01 and by 12.17%/11.23% on RegDB. Code is released at https://github.com/JDAI-CV/CM-NAS.
Fast and Robust Certifiable Estimation of the Relative Pose Between Two Calibrated Cameras
Authors: Mercedes Garcia-Salguero, Javier Gonzalez-Jimenez
Date: 2021-01-21 00:00:00
Tasks: Pose Estimation

This work contributes an efficient algorithm to compute the Relative Pose problem (RPp) between calibrated cameras and certify the optimality of the solution, given a set of pair-wise feature correspondences affected by noise and probably corrupted by wrong matches. We propose a family of certifiers that is shown to increase the ratio of detected optimal solutions. This set of certifiers is incorporated into a fast essential matrix estimation pipeline that, given any initial guess for the RPp, refines it iteratively on the product space of 3D rotations and 2-sphere. In addition, this fast certifiable pipeline is integrated into a robust framework that combines Graduated Non-convexity and the Black-Rangarajan duality between robust functions and line processes. We proved through extensive experiments on synthetic and real data that the proposed framework provides a fast and robust relative pose estimation. We make the code publicly available \url{https://github.com/mergarsal/FastCertRelPose.git}.
Progressive Co-Attention Network for Fine-grained Visual Classification
Authors: Tian Zhang, Dongliang Chang, Zhanyu Ma, Jun Guo
Date: 2021-01-21 00:00:00
Tasks: Classification, Fine-Grained Image Classification, General Classification

Fine-grained visual classification aims to recognize images belonging to multiple sub-categories within a same category. It is a challenging task due to the inherently subtle variations among highly-confused categories. Most existing methods only take an individual image as input, which may limit the ability of models to recognize contrastive clues from different images. In this paper, we propose an effective method called progressive co-attention network (PCA-Net) to tackle this problem. Specifically, we calculate the channel-wise similarity by encouraging interaction between the feature channels within same-category image pairs to capture the common discriminative features. Considering that complementary information is also crucial for recognition, we erase the prominent areas enhanced by the channel interaction to force the network to focus on other discriminative regions. The proposed model has achieved competitive results on three fine-grained visual classification benchmark datasets: CUB-200-2011, Stanford Cars, and FGVC Aircraft.
Eliminate Deviation with Deviation for Data Augmentation and a General Multi-modal Data Learning Method
Authors: Yunpeng Gong, Liqing Huang, Lifei Chen
Date: 2021-01-21 00:00:00
Tasks: Adversarial Defense, Data Augmentation, object-detection, Object Detection, Person Re-Identification, Retrieval

One of the challenges of computer vision is that it needs to adapt to color deviations in changeable environments. Therefore, minimizing the adverse effects of color deviation on the prediction is one of the main goals of vision task. Current solutions focus on using generative models to augment training data to enhance the invariance of input variation. However, such methods often introduce new noise, which limits the gain from generated data. To this end, this paper proposes a strategy eliminate deviation with deviation, which is named Random Color Dropout (RCD). Our hypothesis is that if there are color deviation between the query image and the gallery image, the retrieval results of some examples will be better after ignoring the color information. Specifically, this strategy balances the weights between color features and color-independent features in the neural network by dropouting partial color information in the training data, so as to overcome the effect of color devitaion. The proposed RCD can be combined with various existing ReID models without changing the learning strategy, and can be applied to other computer vision fields, such as object detection. Experiments on several ReID baselines and three common large-scale datasets such as Market1501, DukeMTMC, and MSMT17 have verified the effectiveness of this method. Experiments on Cross-domain tests have shown that this strategy is significant eliminating the domain gap. Furthermore, in order to understand the working mechanism of RCD, we analyzed the effectiveness of this strategy from the perspective of classification, which reveals that it may be better to utilize many instead of all of color information in visual tasks with strong domain variations.
Chest X-ray lung and heart segmentation based on minimal training sets
Authors: Balázs Maga
Date: 2021-01-20 00:00:00
Tasks: Diagnostic, Heart Segmentation

As the COVID-19 pandemic aggravated the excessive workload of doctors globally, the demand for computer aided methods in medical imaging analysis increased even further. Such tools can result in more robust diagnostic pipelines which are less prone to human errors. In our paper, we present a deep neural network to which we refer to as Attention BCDU-Net, and apply it to the task of lung and heart segmentation from chest X-ray (CXR) images, a basic but ardous step in the diagnostic pipeline, for instance for the detection of cardiomegaly. We show that the fine-tuned model exceeds previous state-of-the-art results, reaching $98.1\pm 0.1\%$ Dice score and $95.2\pm 0.1\%$ IoU score on the dataset of Japanese Society of Radiological Technology (JSRT). Besides that, we demonstrate the relative simplicity of the task by attaining surprisingly strong results with training sets of size 10 and 20: in terms of Dice score, $97.0\pm 0.8\%$ and $97.3\pm 0.5$, respectively, while in terms of IoU score, $92.2\pm 1.2\%$ and $93.3\pm 0.4\%$, respectively. To achieve these scores, we capitalize on the mixup augmentation technique, which yields a remarkable gain above $4\%$ IoU score in the size 10 setup.
Learning Ultrasound Rendering from Cross-Sectional Model Slices for Simulated Training
Authors: Lin Zhang, Tiziano Portenier, Orcun Goksel
Date: 2021-01-20 00:00:00
Tasks: Translation

Purpose. Given the high level of expertise required for navigation and interpretation of ultrasound images, computational simulations can facilitate the training of such skills in virtual reality. With ray-tracing based simulations, realistic ultrasound images can be generated. However, due to computational constraints for interactivity, image quality typically needs to be compromised. Methods. We propose herein to bypass any rendering and simulation process at interactive time, by conducting such simulations during a non-time-critical offline stage and then learning image translation from cross-sectional model slices to such simulated frames. We use a generative adversarial framework with a dedicated generator architecture and input feeding scheme, which both substantially improve image quality without increase in network parameters. Integral attenuation maps derived from cross-sectional model slices, texture-friendly strided convolutions, providing stochastic noise and input maps to intermediate layers in order to preserve locality are all shown herein to greatly facilitate such translation task. Results. Given several quality metrics, the proposed method with only tissue maps as input is shown to provide comparable or superior results to a state-of-the-art that uses additional images of low-quality ultrasound renderings. An extensive ablation study shows the need and benefits from the individual contributions utilized in this work, based on qualitative examples and quantitative ultrasound similarity metrics. To that end, a local histogram statistics based error metric is proposed and demonstrated for visualization of local dissimilarities between ultrasound images.
Weighted Fuzzy-Based PSNR for Watermarking
Authors: Maedeh Jamali, Nader Karimi, Shadrokh Samavi
Date: 2021-01-21 00:00:00

One of the problems of conventional visual quality evaluation criteria such as PSNR and MSE is the lack of appropriate standards based on the human visual system (HVS). They are calculated based on the difference of the corresponding pixels in the original and manipulated image. Hence, they practically do not provide a correct understanding of the image quality. Watermarking is an image processing application in which the image's visual quality is an essential criterion for its evaluation. Watermarking requires a criterion based on the HVS that provides more accurate values than conventional measures such as PSNR. This paper proposes a weighted fuzzy-based criterion that tries to find essential parts of an image based on the HVS. Then these parts will have larger weights in computing the final value of PSNR. We compare our results against standard PSNR, and our experiments show considerable consequences.
GhostSR: Learning Ghost Features for Efficient Image Super-Resolution
Authors: Ying Nie, Kai Han, Zhenhua Liu, Chuanjian Liu, Yunhe Wang
Date: 2021-01-21 00:00:00
Tasks: GPU, Image Super-Resolution, Super-Resolution

Modern single image super-resolution (SISR) system based on convolutional neural networks (CNNs) achieves fancy performance while requires huge computational costs. The problem on feature redundancy is well studied in visual recognition task, but rarely discussed in SISR. Based on the observation that many features in SISR models are also similar to each other, we propose to use shift operation to generate the redundant features (i.e., ghost features). Compared with depth-wise convolution which is time-consuming on GPU-like devices, shift operation can bring a practical inference acceleration for CNNs on common hardwares. We analyze the benefits of shift operation on SISR task and make the shift orientation learnable based on Gumbel-Softmax trick. Besides, a clustering procedure is explored based on pre-trained models to identify the intrinsic filters for generating intrinsic features. The ghost features will be derived by moving these intrinsic features along a specific orientation. Finally, the complete output features are constructed by concatenating the intrinsic and ghost features together. Extensive experiments on several benchmark models and datasets demonstrate that both the non-compact and lightweight SISR models embedded with the proposed method can achieve a comparable performance to that of their baselines with a large reduction of parameters, FLOPs and GPU inference latency. For instance, we reduce the parameters by 46%, FLOPs by 46% and GPU inference latency by 42% of $\times2$ EDSR network with basically lossless performance.