pwc-archive/papers-with-abstracts huggingface dataset:
Few-shot Action Recognition with Prototype-centered Attentive Learning
Authors: Xiatian Zhu, Antoine Toisoul, Juan-Manuel Perez-Rua, Li Zhang, Brais Martinez, Tao Xiang
Date: 2021-01-20 00:00:00
Tasks: Action Recognition, Contrastive Learning, Few-Shot action recognition, Few Shot Action Recognition, Fine-grained Action Recognition, Meta-Learning

Few-shot action recognition aims to recognize action classes with few training samples. Most existing methods adopt a meta-learning approach with episodic training. In each episode, the few samples in a meta-training task are split into support and query sets. The former is used to build a classifier, which is then evaluated on the latter using a query-centered loss for model updating. There are however two major limitations: lack of data efficiency due to the query-centered only loss design and inability to deal with the support set outlying samples and inter-class distribution overlapping problems. In this paper, we overcome both limitations by proposing a new Prototype-centered Attentive Learning (PAL) model composed of two novel components. First, a prototype-centered contrastive learning loss is introduced to complement the conventional query-centered learning objective, in order to make full use of the limited training samples in each episode. Second, PAL further integrates a hybrid attentive learning mechanism that can minimize the negative impacts of outliers and promote class separation. Extensive experiments on four standard few-shot action benchmarks show that our method clearly outperforms previous state-of-the-art methods, with the improvement particularly significant (10+\%) on the most challenging fine-grained action recognition benchmark.
Self-supervised pre-training enhances change detection in Sentinel-2 imagery
Authors: Marrit Leenstra, Diego Marcos, Francesca Bovolo, Devis Tuia
Date: 2021-01-20 00:00:00
Tasks: Change Detection, Self-Supervised Learning

While annotated images for change detection using satellite imagery are scarce and costly to obtain, there is a wealth of unlabeled images being generated every day. In order to leverage these data to learn an image representation more adequate for change detection, we explore methods that exploit the temporal consistency of Sentinel-2 times series to obtain a usable self-supervised learning signal. For this, we build and make publicly available (https://zenodo.org/record/4280482) the Sentinel-2 Multitemporal Cities Pairs (S2MTCP) dataset, containing multitemporal image pairs from 1520 urban areas worldwide. We test the results of multiple self-supervised learning methods for pre-training models for change detection and apply it on a public change detection dataset made of Sentinel-2 image pairs (OSCD).
Fooling thermal infrared pedestrian detectors in real world using small bulbs
Authors: Xiaopei Zhu, Xiao Li, Jianmin Li, Zheyao Wang, Xiaolin Hu
Date: 2021-01-20 00:00:00
Tasks: Autonomous Driving, object-detection, Object Detection

Thermal infrared detection systems play an important role in many areas such as night security, autonomous driving, and body temperature detection. They have the unique advantages of passive imaging, temperature sensitivity and penetration. But the security of these systems themselves has not been fully explored, which poses risks in applying these systems. We propose a physical attack method with small bulbs on a board against the state of-the-art pedestrian detectors. Our goal is to make infrared pedestrian detectors unable to detect real-world pedestrians. Towards this goal, we first showed that it is possible to use two kinds of patches to attack the infrared pedestrian detector based on YOLOv3. The average precision (AP) dropped by 64.12% in the digital world, while a blank board with the same size caused the AP to drop by 29.69% only. After that, we designed and manufactured a physical board and successfully attacked YOLOv3 in the real world. In recorded videos, the physical board caused AP of the target detector to drop by 34.48%, while a blank board with the same size caused the AP to drop by 14.91% only. With the ensemble attack techniques, the designed physical board had good transferability to unseen detectors. We also proposed the first physical multispectral (infrared and visible) attack. By using a combination method, we successfully hide from the visible light and infrared object detection systems at the same time.
Focal and Efficient IOU Loss for Accurate Bounding Box Regression
Authors: Yi-Fan Zhang, Weiqiang Ren, Zhang Zhang, Zhen Jia, Liang Wang, Tieniu Tan
Date: 2021-01-20 00:00:00
Tasks: object-detection, Object Detection, Object Localization, regression

In object detection, bounding box regression (BBR) is a crucial step that determines the object localization performance. However, we find that most previous loss functions for BBR have two main drawbacks: (i) Both $\ell_n$-norm and IOU-based loss functions are inefficient to depict the objective of BBR, which leads to slow convergence and inaccurate regression results. (ii) Most of the loss functions ignore the imbalance problem in BBR that the large number of anchor boxes which have small overlaps with the target boxes contribute most to the optimization of BBR. To mitigate the adverse effects caused thereby, we perform thorough studies to exploit the potential of BBR losses in this paper. Firstly, an Efficient Intersection over Union (EIOU) loss is proposed, which explicitly measures the discrepancies of three geometric factors in BBR, i.e., the overlap area, the central point and the side length. After that, we state the Effective Example Mining (EEM) problem and propose a regression version of focal loss to make the regression process focus on high-quality anchor boxes. Finally, the above two parts are combined to obtain a new loss function, namely Focal-EIOU loss. Extensive experiments on both synthetic and real datasets are performed. Notable superiorities on both the convergence speed and the localization accuracy can be achieved over other BBR losses.
Video Relation Detection with Trajectory-aware Multi-modal Features
Authors: Wentao Xie, Guanghui Ren, Si Liu
Date: 2021-01-20 00:00:00
Tasks: Object, object-detection, Object Detection, Relation, Relation Prediction, Video Visual Relation Detection

Video relation detection problem refers to the detection of the relationship between different objects in videos, such as spatial relationship and action relationship. In this paper, we present video relation detection with trajectory-aware multi-modal features to solve this task. Considering the complexity of doing visual relation detection in videos, we decompose this task into three sub-tasks: object detection, trajectory proposal and relation prediction. We use the state-of-the-art object detection method to ensure the accuracy of object trajectory detection and multi-modal feature representation to help the prediction of relation between objects. Our method won the first place on the video relation detection task of Video Relation Understanding Grand Challenge in ACM Multimedia 2020 with 11.74\% mAP, which surpasses other methods by a large margin.
SAR and Optical data fusion based on Anisotropic Diffusion with PCA and Classification using Patch-based with LBP
Authors: Achala Shakya, Mantosh Biswas, Mahesh Pal
Date: 2021-01-20 00:00:00
Tasks: Classification, General Classification

SAR (VV and VH polarization) and optical data are widely used in image fusion to use the complimentary information of each other and to obtain the better-quality image (in terms of spatial and spectral features) for the improved classification results. This paper uses anisotropic diffusion with PCA for the fusion of SAR and optical data and patch-based SVM Classification with LBP (LBP-PSVM). Fusion results with VV polarization performed better than VH polarization using considered fusion method. For classification, the performance of LBP-PSVM using S1 (VV) with S2, S1 (VH) with S2 is compared with SVM classifier (without patch) and PSVM classifier (with patch), respectively. Classification results suggests that the LBP-PSVM classifier is more effective in comparison to SVM and PSVM classifiers for considered data.
An Empirical Study and Analysis on Open-Set Semi-Supervised Learning
Authors: Huixiang Luo, Hao Cheng, Fanxu Meng, Yuting Gao, Ke Li, Mengdan Zhang, Xing Sun
Date: 2021-01-19 00:00:00
Tasks: Data Augmentation

Pseudo-labeling (PL) and Data Augmentation-based Consistency Training (DACT) are two approaches widely used in Semi-Supervised Learning (SSL) methods. These methods exhibit great power in many machine learning tasks by utilizing unlabeled data for efficient training. But in a more realistic setting (termed as open-set SSL), where unlabeled dataset contains out-of-distribution (OOD) samples, the traditional SSL methods suffer severe performance degradation. Recent approaches mitigate the negative influence of OOD samples by filtering them out from the unlabeled data. However, it is not clear whether directly removing the OOD samples is the best choice. Furthermore, why PL and DACT could perform differently in open-set SSL remains a mystery. In this paper, we thoroughly analyze various SSL methods (PL and DACT) on open-set SSL and discuss pros and cons of these two approaches separately. Based on our analysis, we propose Style Disturbance to improve traditional SSL methods on open-set SSL and experimentally show our approach can achieve state-of-the-art results on various datasets by utilizing OOD samples properly. We believe our study can bring new insights for SSL research.
AXM-Net: Implicit Cross-Modal Feature Alignment for Person Re-identification
Authors: Ammarah Farooq, Muhammad Awais, Josef Kittler, Syed Safwan Khalid
Date: 2021-01-19 00:00:00
Tasks: Cross-Modal  Person Re-Identification, Cross-Modal Person Re-Identification, Person Re-Identification, Person Search, Representation Learning, Text based Person Retrieval

Cross-modal person re-identification (Re-ID) is critical for modern video surveillance systems. The key challenge is to align cross-modality representations induced by the semantic information present for a person and ignore background information. This work presents a novel convolutional neural network (CNN) based architecture designed to learn semantically aligned cross-modal visual and textual representations. The underlying building block, named AXM-Block, is a unified multi-layer network that dynamically exploits the multi-scale knowledge from both modalities and re-calibrates each modality according to shared semantics. To complement the convolutional design, contextual attention is applied in the text branch to manipulate long-term dependencies. Moreover, we propose a unique design to enhance visual part-based feature coherence and locality information. Our framework is novel in its ability to implicitly learn aligned semantics between modalities during the feature learning stage. The unified feature learning effectively utilizes textual data as a super-annotation signal for visual representation learning and automatically rejects irrelevant information. The entire AXM-Net is trained end-to-end on CUHK-PEDES data. We report results on two tasks, person search and cross-modal Re-ID. The AXM-Net outperforms the current state-of-the-art (SOTA) methods and achieves 64.44\% Rank@1 on the CUHK-PEDES test set. It also outperforms its competitors by $>$10\% in cross-viewpoint text-to-image Re-ID scenarios on CrossRe-ID and CUHK-SYSU datasets.
Classification of COVID-19 X-ray Images Using a Combination of Deep and Handcrafted Features
Authors: Weihan Zhang, Bryan Pogorelsky, Mark Loveland, Trevor Wolf
Date: 2021-01-19 00:00:00
Tasks: Computed Tomography (CT)

Coronavirus Disease 2019 (COVID-19) demonstrated the need for accurate and fast diagnosis methods for emergent viral diseases. Soon after the emergence of COVID-19, medical practitioners used X-ray and computed tomography (CT) images of patients' lungs to detect COVID-19. Machine learning methods are capable of improving the identification accuracy of COVID-19 in X-ray and CT images, delivering near real-time results, while alleviating the burden on medical practitioners. In this work, we demonstrate the efficacy of a support vector machine (SVM) classifier, trained with a combination of deep convolutional and handcrafted features extracted from X-ray chest scans. We use this combination of features to discriminate between healthy, common pneumonia, and COVID-19 patients. The performance of the combined feature approach is compared with a standard convolutional neural network (CNN) and the SVM trained with handcrafted features. We find that combining the features in our novel framework improves the performance of the classification task compared to the independent application of convolutional and handcrafted features. Specifically, we achieve an accuracy of 0.988 in the classification task with our combined approach compared to 0.963 and 0.983 accuracy for the handcrafted features with SVM and CNN respectively.
Quarter Laplacian Filter for Edge Aware Image Processing
Authors: Yuanhao Gong, Wenming Tang, Lebin Zhou, Lantao Yu, Guoping Qiu
Date: 2021-01-20 00:00:00
Tasks: Image Enhancement, image smoothing, Low-Light Image Enhancement

This paper presents a quarter Laplacian filter that can preserve corners and edges during image smoothing. Its support region is $2\times2$, which is smaller than the $3\times3$ support region of Laplacian filter. Thus, it is more local. Moreover, this filter can be implemented via the classical box filter, leading to high performance for real time applications. Finally, we show its edge preserving property in several image processing tasks, including image smoothing, texture enhancement, and low-light image enhancement. The proposed filter can be adopted in a wide range of image processing applications.
SplitSR: An End-to-End Approach to Super-Resolution on Mobile Devices
Authors: Xin Liu, Yuang Li, Josh Fromm, Yuntao Wang, Ziheng Jiang, Alex Mariakakis, Shwetak Patel
Date: 2021-01-20 00:00:00
Tasks: CPU, Super-Resolution

Super-resolution (SR) is a coveted image processing technique for mobile apps ranging from the basic camera apps to mobile health. Existing SR algorithms rely on deep learning models with significant memory requirements, so they have yet to be deployed on mobile devices and instead operate in the cloud to achieve feasible inference time. This shortcoming prevents existing SR methods from being used in applications that require near real-time latency. In this work, we demonstrate state-of-the-art latency and accuracy for on-device super-resolution using a novel hybrid architecture called SplitSR and a novel lightweight residual block called SplitSRBlock. The SplitSRBlock supports channel-splitting, allowing the residual blocks to retain spatial information while reducing the computation in the channel dimension. SplitSR has a hybrid design consisting of standard convolutional blocks and lightweight residual blocks, allowing people to tune SplitSR for their computational budget. We evaluate our system on a low-end ARM CPU, demonstrating both higher accuracy and up to 5 times faster inference than previous approaches. We then deploy our model onto a smartphone in an app called ZoomSR to demonstrate the first-ever instance of on-device, deep learning-based SR. We conducted a user study with 15 participants to have them assess the perceived quality of images that were post-processed by SplitSR. Relative to bilinear interpolation -- the existing standard for on-device SR -- participants showed a statistically significant preference when looking at both images (Z=-9.270, p<0.01) and text (Z=-6.486, p<0.01).
Improved Signed Distance Function for 2D Real-time SLAM and Accurate Localization
Authors: Xingyin Fu, Zheng Fang, Xizhen Xiao, Yijia He, Xiao Liu
Date: 2021-01-20 00:00:00
Tasks: Pose Estimation

Accurate mapping and localization are very important for many industrial robotics applications. In this paper, we propose an improved Signed Distance Function (SDF) for both 2D SLAM and pure localization to improve the accuracy of mapping and localization. To achieve this goal, firstly we improved the back-end mapping to build a more accurate SDF map by extending the update range and building free space, etc. Secondly, to get more accurate pose estimation for the front-end, we proposed a new iterative registration method to align the current scan to the SDF submap by removing random outliers of laser scanners. Thirdly, we merged all the SDF submaps to produce an integrated SDF map for highly accurate pure localization. Experimental results show that based on the merged SDF map, a localization accuracy of a few millimeters (5mm) can be achieved globally within the map. We believe that this method is important for mobile robots working in scenarios where high localization accuracy matters.
Bridge the Vision Gap from Field to Command: A Deep Learning Network Enhancing Illumination and Details
Authors: Zhuqing Jiang, Chang Liu, Ya'nan Wang, Kai Li, Aidong Men, Haiying Wang, Haiyong Luo
Date: 2021-01-20 00:00:00
Tasks: Image Enhancement, Low-Light Image Enhancement

With the goal of tuning up the brightness, low-light image enhancement enjoys numerous applications, such as surveillance, remote sensing and computational photography. Images captured under low-light conditions often suffer from poor visibility and blur. Solely brightening the dark regions will inevitably amplify the blur, thus may lead to detail loss. In this paper, we propose a simple yet effective two-stream framework named NEID to tune up the brightness and enhance the details simultaneously without introducing many computational costs. Precisely, the proposed method consists of three parts: Light Enhancement (LE), Detail Refinement (DR) and Feature Fusing (FF) module, which can aggregate composite features oriented to multiple tasks based on channel attention mechanism. Extensive experiments conducted on several benchmark datasets demonstrate the efficacy of our method and its superiority over state-of-the-art methods.
Variational Autoencoders with a Structural Similarity Loss in Time of Flight MRAs
Authors: Kimberley M. Timmins, Irene C. van der Schaaf, Ynte M. Ruigrok, Birgitta K. Velthuis, Hugo J. Kuijf
Date: 2021-01-20 00:00:00
Tasks: Anomaly Detection, SSIM

Time-of-Flight Magnetic Resonance Angiographs (TOF-MRAs) enable visualization and analysis of cerebral arteries. This analysis may indicate normal variation of the configuration of the cerebrovascular system or vessel abnormalities, such as aneurysms. A model would be useful to represent normal cerebrovascular structure and variabilities in a healthy population and to differentiate from abnormalities. Current anomaly detection using autoencoding convolutional neural networks usually use a voxelwise mean-error for optimization. We propose optimizing a variational-autoencoder (VAE) with structural similarity loss (SSIM) for TOF-MRA reconstruction. A patch-trained 2D fully-convolutional VAE was optimized for TOF-MRA reconstruction by comparing vessel segmentations of original and reconstructed MRAs. The method was trained and tested on two datasets: the IXI dataset, and a subset from the ADAM challenge. Both trained networks were tested on a dataset including subjects with aneurysms. We compared VAE optimization with L2-loss and SSIM-loss. Performance was evaluated between original and reconstructed MRAs using mean square error, mean-SSIM, peak-signal-to-noise-ratio and dice similarity index (DSI) of segmented vessels. The L2-optimized VAE outperforms SSIM, with improved reconstruction metrics and DSIs for both datasets. Optimization using SSIM performed best for visual image quality, but with discrepancy in quantitative reconstruction and vascular segmentation. The larger, more diverse IXI dataset had overall better performance. Reconstruction metrics, including SSIM, were lower for MRAs including aneurysms. A SSIM-optimized VAE improved the visual perceptive image quality of TOF-MRA reconstructions. A L2-optimized VAE performed best for TOF-MRA reconstruction, where the vascular segmentation is important. SSIM is a potential metric for anomaly detection of MRAs.
Component Tree Loss Function: Definition and Optimization
Authors: Benjamin Perret, Jean Cousty
Date: 2021-01-20 00:00:00
Tasks: BIG-bench Machine Learning

In this article, we propose a method to design loss functions based on component trees which can be optimized by gradient descent algorithms and which are therefore usable in conjunction with recent machine learning approaches such as neural networks. We show how the altitudes associated to the nodes of such hierarchical image representations can be differentiated with respect to the image pixel values. This feature is used to design a generic loss function that can select or discard image maxima based on various attributes such as extinction values. The possibilities of the proposed method are demonstrated on simulated and real image filtering.
Fast deep learning correspondence for neuron tracking and identification in C.elegans using synthetic training
Authors: Xinwei Yu, Matthew S. Creamer, Francesco Randi, Anuj K. Sharma, Scott W. Linderman, Andrew M. Leifer
Date: 2021-01-20 00:00:00
Tasks: Transfer Learning

We present an automated method to track and identify neurons in C. elegans, called "fast Deep Learning Correspondence" or fDLC, based on the transformer network architecture. The model is trained once on empirically derived synthetic data and then predicts neural correspondence across held-out real animals via transfer learning. The same pre-trained model both tracks neurons across time and identifies corresponding neurons across individuals. Performance is evaluated against hand-annotated datasets, including NeuroPAL [1]. Using only position information, the method achieves 80.0% accuracy at tracking neurons within an individual and 65.8% accuracy at identifying neurons across individuals. Accuracy is even higher on a published dataset [2]. Accuracy reaches 76.5% when using color information from NeuroPAL. Unlike previous methods, fDLC does not require straightening or transforming the animal into a canonical coordinate system. The method is fast and predicts correspondence in 10 ms making it suitable for future real-time applications.
Implicit Bias of Linear RNNs
Authors: Melikasadat Emami, Mojtaba Sahraee-Ardakan, Parthe Pandit, Sundeep Rangan, Alyson K. Fletcher
Date: 2021-01-19 00:00:00

Contemporary wisdom based on empirical studies suggests that standard recurrent neural networks (RNNs) do not perform well on tasks requiring long-term memory. However, precise reasoning for this behavior is still unknown. This paper provides a rigorous explanation of this property in the special case of linear RNNs. Although this work is limited to linear RNNs, even these systems have traditionally been difficult to analyze due to their non-linear parameterization. Using recently-developed kernel regime analysis, our main result shows that linear RNNs learned from random initializations are functionally equivalent to a certain weighted 1D-convolutional network. Importantly, the weightings in the equivalent model cause an implicit bias to elements with smaller time lags in the convolution and hence, shorter memory. The degree of this bias depends on the variance of the transition kernel matrix at initialization and is related to the classic exploding and vanishing gradients problem. The theory is validated in both synthetic and real data experiments.