pwc-archive/papers-with-abstracts huggingface dataset:
Full-Duplex Strategy for Video Object Segmentation
Authors: Ge-Peng Ji, Deng-Ping Fan, Keren Fu, Zhe Wu, Jianbing Shen, Ling Shao
Date: 2021-08-06 00:00:00
Proceeding: ICCV 2021 10
Tasks: Object, Object Detection, Salient Object Detection, Segmentation, Semantic Segmentation, Unsupervised Video Object Segmentation, Video Object Segmentation, Video Polyp Segmentation, Video Salient Object Detection, Video Semantic Segmentation

Previous video object segmentation approaches mainly focus on using simplex solutions between appearance and motion, limiting feature collaboration efficiency among and across these two cues. In this work, we study a novel and efficient full-duplex strategy network (FSNet) to address this issue, by considering a better mutual restraint scheme between motion and appearance in exploiting the cross-modal features from the fusion and decoding stage. Specifically, we introduce the relational cross-attention module (RCAM) to achieve bidirectional message propagation across embedding sub-spaces. To improve the model's robustness and update the inconsistent features from the spatial-temporal embeddings, we adopt the bidirectional purification module (BPM) after the RCAM. Extensive experiments on five popular benchmarks show that our FSNet is robust to various challenging scenarios (e.g., motion blur, occlusion) and achieves favourable performance against existing cutting-edges both in the video object segmentation and video salient object detection tasks. The project is publicly available at: https://dpfan.net/FSNet.
ELSED: Enhanced Line SEgment Drawing
Authors: Iago Suárez, José M. Buenaposada, Luis Baumela
Date: 2021-08-06 00:00:00
Tasks: Line Detection, Line Segment Detection

Detecting local features, such as corners, segments or blobs, is the first step in the pipeline of many Computer Vision applications. Its speed is crucial for real time applications. In this paper we present ELSED, the fastest line segment detector in the literature. The key for its efficiency is a local segment growing algorithm that connects gradient aligned pixels in presence of small discontinuities. The proposed algorithm not only runs in devices with very low end hardware, but may also be parametrized to foster the detection of short or longer segments, depending on the task at hand. We also introduce new metrics to evaluate the accuracy and repeatability of segment detectors. In our experiments with different public benchmarks we prove that our method is the most efficient in the literature and quantify the accuracy traded for such gain.
SELM: Siamese Extreme Learning Machine with Application to Face Biometrics
Authors: Wasu Kudisthalert, Kitsuchart Pasupa, Aythami Morales, Julian Fierrez
Date: 2021-08-06 00:00:00
Tasks: Face Verification, Triplet

Extreme Learning Machine is a powerful classification method very competitive existing classification methods. It is extremely fast at training. Nevertheless, it cannot perform face verification tasks properly because face verification tasks require comparison of facial images of two individuals at the same time and decide whether the two faces identify the same person. The structure of Extreme Leaning Machine was not designed to feed two input data streams simultaneously, thus, in 2-input scenarios Extreme Learning Machine methods are normally applied using concatenated inputs. However, this setup consumes two times more computational resources and it is not optimized for recognition tasks where learning a separable distance metric is critical. For these reasons, we propose and develop a Siamese Extreme Learning Machine (SELM). SELM was designed to be fed with two data streams in parallel simultaneously. It utilizes a dual-stream Siamese condition in the extra Siamese layer to transform the data before passing it along to the hidden layer. Moreover, we propose a Gender-Ethnicity-Dependent triplet feature exclusively trained on a variety of specific demographic groups. This feature enables learning and extracting of useful facial features of each group. Experiments were conducted to evaluate and compare the performances of SELM, Extreme Learning Machine, and DCNN. The experimental results showed that the proposed feature was able to perform correct classification at 97.87% accuracy and 99.45% AUC. They also showed that using SELM in conjunction with the proposed feature provided 98.31% accuracy and 99.72% AUC. They outperformed the well-known DCNN and Extreme Leaning Machine methods by a wide margin.
Lung Ultrasound Segmentation and Adaptation between COVID-19 and Community-Acquired Pneumonia
Authors: Harry Mason, Lorenzo Cristoni, Andrew Walden, Roberto Lazzari, Thomas Pulimood, Louis Grandjean, Claudia AM Gandini Wheeler-Kingshott, Yipeng Hu, Zachary MC Baum
Date: 2021-08-06 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Lung ultrasound imaging has been shown effective in detecting typical patterns for interstitial pneumonia, as a point-of-care tool for both patients with COVID-19 and other community-acquired pneumonia (CAP). In this work, we focus on the hyperechoic B-line segmentation task. Using deep neural networks, we automatically outline the regions that are indicative of pathology-sensitive artifacts and their associated sonographic patterns. With a real-world data-scarce scenario, we investigate approaches to utilize both COVID-19 and CAP lung ultrasound data to train the networks; comparing fine-tuning and unsupervised domain adaptation. Segmenting either type of lung condition at inference may support a range of clinical applications during evolving epidemic stages, but also demonstrates value in resource-constrained clinical scenarios. Adapting real clinical data acquired from COVID-19 patients to those from CAP patients significantly improved Dice scores from 0.60 to 0.87 (p < 0.001) and from 0.43 to 0.71 (p < 0.001), on independent COVID-19 and CAP test cases, respectively. It is of practical value that the improvement was demonstrated with only a small amount of data in both training and adaptation data sets, a common constraint for deploying machine learning models in clinical practice. Interestingly, we also report that the inverse adaptation, from labelled CAP data to unlabeled COVID-19 data, did not demonstrate an improvement when tested on either condition. Furthermore, we offer a possible explanation that correlates the segmentation performance to label consistency and data domain diversity in this point-of-care lung ultrasound application.
Contrastive Learning for View Classification of Echocardiograms
Authors: Agisilaos Chartsias, Shan Gao, Angela Mumith, Jorge Oliveira, Kanwal Bhatia, Bernhard Kainz, Arian Beqiri
Date: 2021-08-06 00:00:00
Tasks: Classification, Contrastive Learning

Analysis of cardiac ultrasound images is commonly performed in routine clinical practice for quantification of cardiac function. Its increasing automation frequently employs deep learning networks that are trained to predict disease or detect image features. However, such models are extremely data-hungry and training requires labelling of many thousands of images by experienced clinicians. Here we propose the use of contrastive learning to mitigate the labelling bottleneck. We train view classification models for imbalanced cardiac ultrasound datasets and show improved performance for views/classes for which minimal labelled data is available. Compared to a naive baseline model, we achieve an improvement in F1 score of up to 26% in those views while maintaining state-of-the-art performance for the views with sufficiently many labelled training observations.
Uncertainty-Based Dynamic Graph Neighborhoods For Medical Segmentation
Authors: Ufuk Demir, Atahan Ozer, Yusuf H. Sahin, Gozde Unal
Date: 2021-08-06 00:00:00
Tasks: Computed Tomography (CT), Medical Image Analysis, Pancreas Segmentation, Segmentation

In recent years, deep learning based methods have shown success in essential medical image analysis tasks such as segmentation. Post-processing and refining the results of segmentation is a common practice to decrease the misclassifications originating from the segmentation network. In addition to widely used methods like Conditional Random Fields (CRFs) which focus on the structure of the segmented volume/area, a graph-based recent approach makes use of certain and uncertain points in a graph and refines the segmentation according to a small graph convolutional network (GCN). However, there are two drawbacks of the approach: most of the edges in the graph are assigned randomly and the GCN is trained independently from the segmentation network. To address these issues, we define a new neighbor-selection mechanism according to feature distances and combine the two networks in the training procedure. According to the experimental results on pancreas segmentation from Computed Tomography (CT) images, we demonstrate improvement in the quantitative measures. Also, examining the dynamic neighbors created by our method, edges between semantically similar image parts are observed. The proposed method also shows qualitative enhancements in the segmentation maps, as demonstrated in the visual results.
TS4Net: Two-Stage Sample Selective Strategy for Rotating Object Detection
Authors: Kai Feng, Weixing Li, Jun Han, Feng Pan, Dongdong Zheng
Date: 2021-08-06 00:00:00
Tasks: Object, Object Counting, object-detection, Object Detection, regression, Vocal Bursts Valence Prediction

Rotating object detection has wide applications in aerial photographs, remote sensing images, UAVs, etc. At present, most of the rotating object detection datasets focus on the field of remote sensing, and these images are usually shot in high-altitude scenes. However, image datasets captured at low-altitude areas also should be concerned, such as drone-based datasets. So we present a low-altitude dronebased dataset, named UAV-ROD, aiming to promote the research and development in rotating object detection and UAV applications. The UAV-ROD consists of 1577 images and 30,090 instances of car category annotated by oriented bounding boxes. In particular, The UAV-ROD can be utilized for the rotating object detection, vehicle orientation recognition and object counting tasks. Compared with horizontal object detection, the regression stage of the rotation detection is a tricky problem. In this paper, we propose a rotating object detector TS4Net, which contains anchor refinement module (ARM) and two-stage sample selective strategy (TS4). The ARM can convert preseted horizontal anchors into high-quality rotated anchors through twostage anchor refinement. The TS4 module utilizes different constrained sample selective strategies to allocate positive and negative samples, which is adaptive to the regression task in different stages. Benefiting from the ARM and TS4, the TS4Net can achieve superior performance for rotating object detection solely with one preseted horizontal anchor. Extensive experimental results on UAV-ROD dataset and three remote sensing datasets DOTA, HRSC2016 and UCAS-AOD demonstrate that our method achieves competitive performance against most state-of-the-art methods.
STR-GQN: Scene Representation and Rendering for Unknown Cameras Based on Spatial Transformation Routing
Authors: Wen-Cheng Chen, Min-Chun Hu, Chu-Song Chen
Date: 2021-08-06 00:00:00
Proceeding: ICCV 2021 10

Geometry-aware modules are widely applied in recent deep learning architectures for scene representation and rendering. However, these modules require intrinsic camera information that might not be obtained accurately. In this paper, we propose a Spatial Transformation Routing (STR) mechanism to model the spatial properties without applying any geometric prior. The STR mechanism treats the spatial transformation as the message passing process, and the relation between the view poses and the routing weights is modeled by an end-to-end trainable neural network. Besides, an Occupancy Concept Mapping (OCM) framework is proposed to provide explainable rationals for scene-fusion processes. We conducted experiments on several datasets and show that the proposed STR mechanism improves the performance of the Generative Query Network (GQN). The visualization results reveal that the routing process can pass the observed information from one location of some view to the associated location in the other view, which demonstrates the advantage of the proposed model in terms of spatial cognition.
Spatiotemporal Contrastive Learning of Facial Expressions in Videos
Authors: Shuvendu Roy, Ali Etemad
Date: 2021-08-06 00:00:00
Tasks: Contrastive Learning, Facial Expression Recognition, Facial Expression Recognition (FER)

We propose a self-supervised contrastive learning approach for facial expression recognition (FER) in videos. We propose a novel temporal sampling-based augmentation scheme to be utilized in addition to standard spatial augmentations used for contrastive learning. Our proposed temporal augmentation scheme randomly picks from one of three temporal sampling techniques: (1) pure random sampling, (2) uniform sampling, and (3) sequential sampling. This is followed by a combination of up to three standard spatial augmentations. We then use a deep R(2+1)D network for FER, which we train in a self-supervised fashion based on the augmentations and subsequently fine-tune. Experiments are performed on the Oulu-CASIA dataset and the performance is compared to other works in FER. The results indicate that our method achieves an accuracy of 89.4%, setting a new state-of-the-art by outperforming other works. Additional experiments and analysis confirm the considerable contribution of the proposed temporal augmentation versus the existing spatial ones.
Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer
Authors: Zhihe Lu, Sen He, Xiatian Zhu, Li Zhang, Yi-Zhe Song, Tao Xiang
Date: 2021-08-06 00:00:00
Proceeding: ICCV 2021 10
Tasks: Decoder, Few-Shot Semantic Segmentation, Meta-Learning, Semantic Segmentation

A few-shot semantic segmentation model is typically composed of a CNN encoder, a CNN decoder and a simple classifier (separating foreground and background pixels). Most existing methods meta-learn all three model components for fast adaptation to a new class. However, given that as few as a single support set image is available, effective model adaption of all three components to the new class is extremely challenging. In this work we propose to simplify the meta-learning task by focusing solely on the simplest component, the classifier, whilst leaving the encoder and decoder to pre-training. We hypothesize that if we pre-train an off-the-shelf segmentation model over a set of diverse training classes with sufficient annotations, the encoder and decoder can capture rich discriminative features applicable for any unseen classes, rendering the subsequent meta-learning stage unnecessary. For the classifier meta-learning, we introduce a Classifier Weight Transformer (CWT) designed to dynamically adapt the supportset trained classifier's weights to each query image in an inductive way. Extensive experiments on two standard benchmarks show that despite its simplicity, our method outperforms the state-of-the-art alternatives, often by a large margin.Code is available on https://github.com/zhiheLu/CWT-for-FSS.
Road Scenes Segmentation Across Different Domains by Disentangling Latent Representations
Authors: Francesco Barbato, Umberto Michieli, Marco Toldo, Pietro Zanuttigh
Date: 2021-08-06 00:00:00
Tasks: Domain Adaptation, Semantic Segmentation

Deep learning models obtain impressive accuracy in road scenes understanding, however they need a large quantity of labeled samples for their training. Additionally, such models do not generalise well to environments where the statistical properties of data do not perfectly match those of training scenes, and this can be a significant problem for intelligent vehicles. Hence, domain adaptation approaches have been introduced to transfer knowledge acquired on a label-abundant source domain to a related label-scarce target domain. In this work, we design and carefully analyse multiple latent space-shaping regularisation strategies that work together to reduce the domain shift. More in detail, we devise a feature clustering strategy to increase domain alignment, a feature perpendicularity constraint to space apart features belonging to different semantic classes, including those not present in the current batch, and a feature norm alignment strategy to separate active and inactive channels. In addition, we propose a novel evaluation metric to capture the relative performance of an adapted model with respect to supervised training. We validate our framework in driving scenarios, considering both synthetic-to-real and real-to-real adaptation, outperforming previous feature-level state-of-the-art methods on multiple road scenes benchmarks.
Feature Detection for Hand Hygiene Stages
Authors: Rashmi Bakshi, Jane Courtney, Damon Berry, Graham Gavin
Date: 2021-08-06 00:00:00

The process of hand washing involves complex hand movements. There are six principal sequential steps for washing hands as per the World Health Organisation (WHO) guidelines. In this work, a detailed description of an aluminium rig construction for creating a robust hand-washing dataset is discussed. The preliminary results with the help of image processing and computer vision algorithms for hand pose extraction and feature detection such as Harris detector, Shi-Tomasi and SIFT are demonstrated. The hand hygiene pose- Rub hands palm to palm was captured as an input image for running all the experiments. The future work will focus upon processing the video recordings of hand movements captured and applying deep-learning solutions for the classification of hand-hygiene stages.
MmWave Radar and Vision Fusion for Object Detection in Autonomous Driving: A Review
Authors: Zhiqing Wei, Fengkai Zhang, Shuo Chang, Yangyang Liu, Huici Wu, Zhiyong Feng
Date: 2021-08-06 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, Object, object-detection, Object Detection, Sensor Fusion

With autonomous driving developing in a booming stage, accurate object detection in complex scenarios attract wide attention to ensure the safety of autonomous driving. Millimeter wave (mmWave) radar and vision fusion is a mainstream solution for accurate obstacle detection. This article presents a detailed survey on mmWave radar and vision fusion based obstacle detection methods. First, we introduce the tasks, evaluation criteria, and datasets of object detection for autonomous driving. The process of mmWave radar and vision fusion is then divided into three parts: sensor deployment, sensor calibration, and sensor fusion, which are reviewed comprehensively. Specifically, we classify the fusion methods into data level, decision level, and feature level fusion methods. In addition, we introduce three-dimensional(3D) object detection, the fusion of lidar and vision in autonomous driving and multimodal information fusion, which are promising for the future. Finally, we summarize this article.
Two New Low Rank Tensor Completion Methods Based on Sum Nuclear Norm
Authors: HongBing Zhang, Xinyi Liu, HongTao Fan, YaJing Li, Yinlin Ye
Date: 2021-08-06 00:00:00

The low rank tensor completion (LRTC) problem has attracted great attention in computer vision and signal processing. How to acquire high quality image recovery effect is still an urgent task to be solved at present. This paper proposes a new tensor $L_{2,1}$ norm minimization model (TLNM) that integrates sum nuclear norm (SNN) method, differing from the classical tensor nuclear norm (TNN)-based tensor completion method, with $L_{2,1}$ norm and Qatar Riyal decomposition for solving the LRTC problem. To improve the utilization rate of the local prior information of the image, a total variation (TV) regularization term is introduced, resulting in a new class of tensor $L_{2,1}$ norm minimization with total variation model (TLNMTV). Both proposed models are convex and therefore have global optimal solutions. Moreover, we adopt the Alternating Direction Multiplier Method (ADMM) to obtain the closed-form solution of each variable, thus ensuring the feasibility of the algorithm. Numerical experiments show that the two proposed algorithms are convergent and outperform compared methods. In particular, our method significantly outperforms the contrastive methods when the sampling rate of hyperspectral images is 2.5\%.
Efficient and Generic Interactive Segmentation Framework to Correct Mispredictions during Clinical Evaluation of Medical Images
Authors: Bhavani Sambaturu, Ashutosh Gupta, C. V. Jawahar, Chetan Arora
Date: 2021-08-06 00:00:00
Tasks: Brain Segmentation, Image Segmentation, Interactive Segmentation, Segmentation, Semantic Segmentation

Semantic segmentation of medical images is an essential first step in computer-aided diagnosis systems for many applications. However, given many disparate imaging modalities and inherent variations in the patient data, it is difficult to consistently achieve high accuracy using modern deep neural networks (DNNs). This has led researchers to propose interactive image segmentation techniques where a medical expert can interactively correct the output of a DNN to the desired accuracy. However, these techniques often need separate training data with the associated human interactions, and do not generalize to various diseases, and types of medical images. In this paper, we suggest a novel conditional inference technique for DNNs which takes the intervention by a medical expert as test time constraints and performs inference conditioned upon these constraints. Our technique is generic can be used for medical images from any modality. Unlike other methods, our approach can correct multiple structures simultaneously and add structures missed at initial segmentation. We report an improvement of 13.3, 12.5, 17.8, 10.2, and 12.4 times in user annotation time than full human annotation for the nucleus, multiple cells, liver and tumor, organ, and brain segmentation respectively. We report a time saving of 2.8, 3.0, 1.9, 4.4, and 8.6 fold compared to other interactive segmentation techniques. Our method can be useful to clinicians for diagnosis and post-surgical follow-up with minimal intervention from the medical expert. The source-code and the detailed results are available here [1].
Improving Contrastive Learning by Visualizing Feature Transformation
Authors: Rui Zhu, Bingchen Zhao, Jingen Liu, Zhenglong Sun, Chang Wen Chen
Date: 2021-08-06 00:00:00
Proceeding: ICCV 2021 10
Tasks: Contrastive Learning, Data Augmentation, POS, Self-Supervised Learning

Contrastive learning, which aims at minimizing the distance between positive pairs while maximizing that of negative ones, has been widely and successfully applied in unsupervised feature learning, where the design of positive and negative (pos/neg) pairs is one of its keys. In this paper, we attempt to devise a feature-level data manipulation, differing from data augmentation, to enhance the generic contrastive self-supervised learning. To this end, we first design a visualization scheme for pos/neg score (Pos/neg score indicates cosine similarity of pos/neg pair.) distribution, which enables us to analyze, interpret and understand the learning process. To our knowledge, this is the first attempt of its kind. More importantly, leveraging this tool, we gain some significant observations, which inspire our novel Feature Transformation proposals including the extrapolation of positives. This operation creates harder positives to boost the learning because hard positives enable the model to be more view-invariant. Besides, we propose the interpolation among negatives, which provides diversified negatives and makes the model more discriminative. It is the first attempt to deal with both challenges simultaneously. Experiment results show that our proposed Feature Transformation can improve at least 6.0% accuracy on ImageNet-100 over MoCo baseline, and about 2.0% accuracy on ImageNet-1K over the MoCoV2 baseline. Transferring to the downstream tasks successfully demonstrate our model is less task-bias. Visualization tools and codes https://github.com/DTennant/CL-Visualizing-Feature-Transformation .