pwc-archive/papers-with-abstracts huggingface dataset:
Learning content and context with language bias for Visual Question Answering
Authors: Chao Yang, Su Feng, Dongsheng Li, HuaWei Shen, Guoqing Wang, Bin Jiang
Date: 2020-12-21 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Visual Question Answering (VQA) is a challenging multimodal task to answer questions about an image. Many works concentrate on how to reduce language bias which makes models answer questions ignoring visual content and language context. However, reducing language bias also weakens the ability of VQA models to learn context prior. To address this issue, we propose a novel learning strategy named CCB, which forces VQA models to answer questions relying on Content and Context with language Bias. Specifically, CCB establishes Content and Context branches on top of a base VQA model and forces them to focus on local key content and global effective context respectively. Moreover, a joint loss function is proposed to reduce the importance of biased samples and retain their beneficial influence on answering questions. Experiments show that CCB outperforms the state-of-the-art methods in terms of accuracy on VQA-CP v2.
Automated segmentation of an intensity calibration phantom in clinical CT images using a convolutional neural network
Authors: Keisuke Uemura, Yoshito Otake, Masaki Takao, Mazen Soufi, Akihiro Kawasaki, Nobuhiko Sugano, Yoshinobu Sato
Date: 2020-12-21 00:00:00
Tasks: Computed Tomography (CT)

Purpose: To apply a convolutional neural network (CNN) to develop a system that segments intensity calibration phantom regions in computed tomography (CT) images, and to test the system in a large cohort to evaluate its robustness. Methods: A total of 1040 cases (520 cases each from two institutions), in which an intensity calibration phantom (B-MAS200, Kyoto Kagaku, Kyoto, Japan) was used, were included herein. A training dataset was created by manually segmenting the regions of the phantom for 40 cases (20 cases each). Segmentation accuracy of the CNN model was assessed with the Dice coefficient and the average symmetric surface distance (ASD) through the 4-fold cross validation. Further, absolute differences of radiodensity values (in Hounsfield units: HU) were compared between manually segmented regions and automatically segmented regions. The system was tested on the remaining 1000 cases. For each institution, linear regression was applied to calculate coefficients for the correlation between radiodensity and the densities of the phantom. Results: After training, the median Dice coefficient was 0.977, and the median ASD was 0.116 mm. When segmented regions were compared between manual segmentation and automated segmentation, the median absolute difference was 0.114 HU. For the test cases, the median correlation coefficient was 0.9998 for one institution and was 0.9999 for the other, with a minimum value of 0.9863. Conclusions: The CNN model successfully segmented the calibration phantom's regions in the CT images with excellent accuracy, and the automated method was found to be at least equivalent to the conventional manual method. Future study should integrate the system by automatically segmenting the region of interest in bones such that the bone mineral density can be fully automatically quantified from CT images.
Optimizing Deep Neural Networks through Neuroevolution with Stochastic Gradient Descent
Authors: Haichao Zhang, Kuangrong Hao, Lei Gao, Bing Wei, Xuesong Tang
Date: 2020-12-21 00:00:00

Deep neural networks (DNNs) have achieved remarkable success in computer vision; however, training DNNs for satisfactory performance remains challenging and suffers from sensitivity to empirical selections of an optimization algorithm for training. Stochastic gradient descent (SGD) is dominant in training a DNN by adjusting neural network weights to minimize the DNNs loss function. As an alternative approach, neuroevolution is more in line with an evolutionary process and provides some key capabilities that are often unavailable in SGD, such as the heuristic black-box search strategy based on individual collaboration in neuroevolution. This paper proposes a novel approach that combines the merits of both neuroevolution and SGD, enabling evolutionary search, parallel exploration, and an effective probe for optimal DNNs. A hierarchical cluster-based suppression algorithm is also developed to overcome similar weight updates among individuals for improving population diversity. We implement the proposed approach in four representative DNNs based on four publicly-available datasets. Experiment results demonstrate that the four DNNs optimized by the proposed approach all outperform corresponding ones optimized by only SGD on all datasets. The performance of DNNs optimized by the proposed approach also outperforms state-of-the-art deep networks. This work also presents a meaningful attempt for pursuing artificial general intelligence.
Diverse Knowledge Distillation for End-to-End Person Search
Authors: Xinyu Zhang, Xinlong Wang, Jia-Wang Bian, Chunhua Shen, Mingyu You
Date: 2020-12-21 00:00:00
Tasks: Human Detection, Knowledge Distillation, Person Search

Person search aims to localize and identify a specific person from a gallery of images. Recent methods can be categorized into two groups, i.e., two-step and end-to-end approaches. The former views person search as two independent tasks and achieves dominant results using separately trained person detection and re-identification (Re-ID) models. The latter performs person search in an end-to-end fashion. Although the end-to-end approaches yield higher inference efficiency, they largely lag behind those two-step counterparts in terms of accuracy. In this paper, we argue that the gap between the two kinds of methods is mainly caused by the Re-ID sub-networks of end-to-end methods. To this end, we propose a simple yet strong end-to-end network with diverse knowledge distillation to break the bottleneck. We also design a spatial-invariant augmentation to assist model to be invariant to inaccurate detection results. Experimental results on the CUHK-SYSU and PRW datasets demonstrate the superiority of our method against existing approaches -- it achieves on par accuracy with state-of-the-art two-step methods while maintaining high efficiency due to the single joint model. Code is available at: https://git.io/DKD-PersonSearch.
Searching for Controllable Image Restoration Networks
Authors: Heewon Kim, Sungyong Baik, Myungsub Choi, Janghoon Choi, Kyoung Mu Lee
Date: 2020-12-21 00:00:00
Proceeding: ICCV 2021 10
Tasks: 4k, GPU, Image Restoration, Neural Architecture Search

Diverse user preferences over images have recently led to a great amount of interest in controlling the imagery effects for image restoration tasks. However, existing methods require separate inference through the entire network per each output, which hinders users from readily comparing multiple imagery effects due to long latency. To this end, we propose a novel framework based on a neural architecture search technique that enables efficient generation of multiple imagery effects via two stages of pruning: task-agnostic and task-specific pruning. Specifically, task-specific pruning learns to adaptively remove the irrelevant network parameters for each task, while task-agnostic pruning learns to find an efficient architecture by sharing the early layers of the network across different tasks. Since the shared layers allow for feature reuse, only a single inference of the task-agnostic layers is needed to generate multiple imagery effects from the input image. Using the proposed task-agnostic and task-specific pruning schemes together significantly reduces the FLOPs and the actual latency of inference compared to the baseline. We reduce 95.7% of the FLOPs when generating 27 imagery effects, and make the GPU latency 73.0% faster on 4K-resolution images.
DAQ: Channel-Wise Distribution-Aware Quantization for Deep Image Super-Resolution Networks
Authors: Cheeun Hong, Heewon Kim, Sungyong Baik, Junghun Oh, Kyoung Mu Lee
Date: 2020-12-21 00:00:00
Tasks: Image Super-Resolution, Quantization, Super-Resolution

Quantizing deep convolutional neural networks for image super-resolution substantially reduces their computational costs. However, existing works either suffer from a severe performance drop in ultra-low precision of 4 or lower bit-widths, or require a heavy fine-tuning process to recover the performance. To our knowledge, this vulnerability to low precisions relies on two statistical observations of feature map values. First, distribution of feature map values varies significantly per channel and per input image. Second, feature maps have outliers that can dominate the quantization error. Based on these observations, we propose a novel distribution-aware quantization scheme (DAQ) which facilitates accurate training-free quantization in ultra-low precision. A simple function of DAQ determines dynamic range of feature maps and weights with low computational burden. Furthermore, our method enables mixed-precision quantization by calculating the relative sensitivity of each channel, without any training process involved. Nonetheless, quantization-aware training is also applicable for auxiliary performance gain. Our new method outperforms recent training-free and even training-based quantization methods to the state-of-the-art image super-resolution networks in ultra-low precision.
Image Annotation based on Deep Hierarchical Context Networks
Authors: Mingyuan Jiu, Hichem Sahbi
Date: 2020-12-21 00:00:00
Tasks: Representation Learning

Context modeling is one of the most fertile subfields of visual recognition which aims at designing discriminant image representations while incorporating their intrinsic and extrinsic relationships. However, the potential of context modeling is currently underexplored and most of the existing solutions are either context-free or restricted to simple handcrafted geometric relationships. We introduce in this paper DHCN: a novel Deep Hierarchical Context Network that leverages different sources of contexts including geometric and semantic relationships. The proposed method is based on the minimization of an objective function mixing a fidelity term, a context criterion and a regularizer. The solution of this objective function defines the architecture of a bi-level hierarchical context network; the first level of this network captures scene geometry while the second one corresponds to semantic relationships. We solve this representation learning problem by training its underlying deep network whose parameters correspond to the most influencing bi-level contextual relationships and we evaluate its performances on image annotation using the challenging ImageCLEF benchmark.
Unsupervised Domain Adaptation with Temporal-Consistent Self-Training for 3D Hand-Object Joint Reconstruction
Authors: Mengshi Qi, Edoardo Remelli, Mathieu Salzmann, Pascal Fua
Date: 2020-12-21 00:00:00
Tasks: Domain Adaptation, Generative Adversarial Network, Unsupervised Domain Adaptation

Deep learning-solutions for hand-object 3D pose and shape estimation are now very effective when an annotated dataset is available to train them to handle the scenarios and lighting conditions they will encounter at test time. Unfortunately, this is not always the case, and one often has to resort to training them on synthetic data, which does not guarantee that they will work well in real situations. In this paper, we introduce an effective approach to addressing this challenge by exploiting 3D geometric constraints within a cycle generative adversarial network (CycleGAN) to perform domain adaptation. Furthermore, in contrast to most existing works, which fail to leverage the rich temporal information available in unlabeled real videos as a source of supervision, we propose to enforce short- and long-term temporal consistency to fine-tune the domain-adapted model in a self-supervised fashion. We will demonstrate that our approach outperforms state-of-the-art 3D hand-object joint reconstruction methods on three widely-used benchmarks and will make our code publicly available.
Centralized Information Interaction for Salient Object Detection
Authors: Jiang-Jiang Liu, Zhi-Ang Liu, Ming-Ming Cheng
Date: 2020-12-21 00:00:00
Tasks: Object, object-detection, Object Detection, Salient Object Detection, Spatial Interpolation

The U-shape structure has shown its advantage in salient object detection for efficiently combining multi-scale features. However, most existing U-shape based methods focused on improving the bottom-up and top-down pathways while ignoring the connections between them. This paper shows that by centralizing these connections, we can achieve the cross-scale information interaction among them, hence obtaining semantically stronger and positionally more precise features. To inspire the potential of the newly proposed strategy, we further design a relative global calibration module that can simultaneously process multi-scale inputs without spatial interpolation. Benefiting from the above strategy and module, our proposed approach can aggregate features more effectively while introducing only a few additional parameters. Our approach can cooperate with various existing U-shape-based salient object detection methods by substituting the connections between the bottom-up and top-down pathways. Experimental results demonstrate that our proposed approach performs favorably against the previous state-of-the-arts on five widely used benchmarks with less computational complexity. The source code will be publicly available.
Monocular Depth Parameterizing Networks
Authors: Patrik Persson, Linn Öström, Carl Olsson
Date: 2020-12-21 00:00:00
Tasks: Depth Estimation, Monocular Depth Estimation

Monocular depth estimation is a highly challenging problem that is often addressed with deep neural networks. While these are able to use recognition of image features to predict reasonably looking depth maps the result often has low metric accuracy. In contrast traditional stereo methods using multiple cameras provide highly accurate estimation when pixel matching is possible. In this work we propose to combine the two approaches leveraging their respective strengths. For this purpose we propose a network structure that given an image provides a parameterization of a set of depth maps with feasible shapes. Optimizing over the parameterization then allows us to search the shapes for a photo consistent solution with respect to other images. This allows us to enforce geometric properties that are difficult to observe in single image as well as relaxes the learning problem allowing us to use relatively small networks. Our experimental evaluation shows that our method generates more accurate depth maps and generalizes better than competing state-of-the-art approaches.
PBNS: Physically Based Neural Simulator for Unsupervised Garment Pose Space Deformation
Authors: Hugo Bertiche, Meysam Madadi, Sergio Escalera
Date: 2020-12-21 00:00:00
Tasks: Physical Simulations

We present a methodology to automatically obtain Pose Space Deformation (PSD) basis for rigged garments through deep learning. Classical approaches rely on Physically Based Simulations (PBS) to animate clothes. These are general solutions that, given a sufficiently fine-grained discretization of space and time, can achieve highly realistic results. However, they are computationally expensive and any scene modification prompts the need of re-simulation. Linear Blend Skinning (LBS) with PSD offers a lightweight alternative to PBS, though, it needs huge volumes of data to learn proper PSD. We propose using deep learning, formulated as an implicit PBS, to unsupervisedly learn realistic cloth Pose Space Deformations in a constrained scenario: dressed humans. Furthermore, we show it is possible to train these models in an amount of time comparable to a PBS of a few sequences. To the best of our knowledge, we are the first to propose a neural simulator for cloth. While deep-based approaches in the domain are becoming a trend, these are data-hungry models. Moreover, authors often propose complex formulations to better learn wrinkles from PBS data. Supervised learning leads to physically inconsistent predictions that require collision solving to be used. Also, dependency on PBS data limits the scalability of these solutions, while their formulation hinders its applicability and compatibility. By proposing an unsupervised methodology to learn PSD for LBS models (3D animation standard), we overcome both of these drawbacks. Results obtained show cloth-consistency in the animated garments and meaningful pose-dependant folds and wrinkles. Our solution is extremely efficient, handles multiple layers of cloth, allows unsupervised outfit resizing and can be easily applied to any custom 3D avatar.
Accurate Object Association and Pose Updating for Semantic SLAM
Authors: Kaiqi Chen, Jialing Liu, Jianhua Zhang, Zhenhua Wang
Date: 2020-12-21 00:00:00
Tasks: Multiple Object Tracking, Object, Object Tracking, Semantic SLAM

Nowadays in the field of semantic SLAM, how to correctly use semantic information for data association is still a problem worthy of study. The key to solving this problem is to correctly associate multiple object measurements of one object landmark, and refine the pose of object landmark. However, different objects locating closely are prone to be associated as one object landmark, and it is difficult to pick up a best pose from multiple object measurements associated with one object landmark. To tackle these problems, we propose a hierarchical object association strategy by means of multiple object tracking, through which closing objects will be correctly associated to different object landmarks, and an approach to refine the pose of object landmark from multiple object measurements. The proposed method is evaluated on a simulated sequence and several sequences in the Kitti dataset. Experimental results show a very impressive improvement with respect to the traditional SLAM and the state-of-the-art semantic SLAM method.
Knowledge Transfer Based Fine-grained Visual Classification
Authors: Siqing Zhang, Ruoyi Du, Dongliang Chang, Zhanyu Ma, Jun Guo
Date: 2020-12-21 00:00:00
Tasks: Classification, Fine-Grained Image Classification, General Classification, Transfer Learning

Fine-grained visual classification (FGVC) aims to distinguish the sub-classes of the same category and its essential solution is to mine the subtle and discriminative regions. Convolution neural networks (CNNs), which employ the cross entropy loss (CE-loss) as the loss function, show poor performance since the model can only learn the most discriminative part and ignore other meaningful regions. Some existing works try to solve this problem by mining more discriminative regions by some detection techniques or attention mechanisms. However, most of them will meet the background noise problem when trying to find more discriminative regions. In this paper, we address it in a knowledge transfer learning manner. Multiple models are trained one by one, and all previously trained models are regarded as teacher models to supervise the training of the current one. Specifically, a orthogonal loss (OR-loss) is proposed to encourage the network to find diverse and meaningful regions. In addition, the first model is trained with only CE-Loss. Finally, all models' outputs with complementary knowledge are combined together for the final prediction result. We demonstrate the superiority of the proposed method and obtain state-of-the-art (SOTA) performances on three popular FGVC datasets.
3D Object Detection with Pointformer
Authors: Xuran Pan, Zhuofan Xia, Shiji Song, Li Erran Li, Gao Huang
Date: 2020-12-21 00:00:00
Proceeding: CVPR 2021 1
Tasks: 3D Object Detection, Object, object-detection, Object Detection, Object Proposal Generation

Feature learning for 3D object detection from point clouds is very challenging due to the irregularity of 3D point cloud data. In this paper, we propose Pointformer, a Transformer backbone designed for 3D point clouds to learn features effectively. Specifically, a Local Transformer module is employed to model interactions among points in a local region, which learns context-dependent region features at an object level. A Global Transformer is designed to learn context-aware representations at the scene level. To further capture the dependencies among multi-scale representations, we propose Local-Global Transformer to integrate local features with global features from higher resolution. In addition, we introduce an efficient coordinate refinement module to shift down-sampled points closer to object centroids, which improves object proposal generation. We use Pointformer as the backbone for state-of-the-art object detection models and demonstrate significant improvements over original models on both indoor and outdoor datasets.
Exploiting Vulnerability of Pooling in Convolutional Neural Networks by Strict Layer-Output Manipulation for Adversarial Attacks
Authors: Chenchen Zhao, Hao Li
Date: 2020-12-21 00:00:00
Tasks: Adversarial Attack, Dimensionality Reduction

Convolutional neural networks (CNN) have been more and more applied in mobile robotics such as intelligent vehicles. Security of CNNs in robotics applications is an important issue, for which potential adversarial attacks on CNNs are worth research. Pooling is a typical step of dimension reduction and information discarding in CNNs. Such information discarding may result in mis-deletion and mis-preservation of data features which largely influence the output of the network. This may aggravate the vulnerability of CNNs to adversarial attacks. In this paper, we conduct adversarial attacks on CNNs from the perspective of network structure by investigating and exploiting the vulnerability of pooling. First, a novel adversarial attack methodology named Strict Layer-Output Manipulation (SLOM) is proposed. Then an attack method based on Strict Pooling Manipulation (SPM) which is an instantiation of the SLOM spirit is designed to effectively realize both type I and type II adversarial attacks on a target CNN. Performances of attacks based on SPM at different depths are also investigated and compared. Moreover, performances of attack methods designed by instantiating the SLOM spirit with different operation layers of CNNs are compared. Experiment results reflect that pooling tends to be more vulnerable to adversarial attacks than other operations in CNNs.
Amplifying the Anterior-Posterior Difference via Data Enhancement -- A More Robust Deep Monocular Orientation Estimation Solution
Authors: Chenchen Zhao, Hao Li
Date: 2020-12-21 00:00:00
Tasks: Decision Making

Existing deep-learning based monocular orientation estimation algorithms faces the problem of confusion between the anterior and posterior parts of the objects, caused by the feature similarity of such parts in typical objects in traffic scenes such as cars and pedestrians. While difficult to solve, the problem may lead to serious orientation estimation errors, and pose threats to the upcoming decision making process of the ego vehicle, since the predicted tracks of objects may have directions opposite to ground truths. In this paper, we mitigate this problem by proposing a pretraining method. The method focuses on predicting the left/right semicircle in which the orientation of the object is located. The trained semicircle prediction model is then integrated into the orientation angle estimation model which predicts a value in range $[0, \pi]$. Experiment results show that the proposed semicircle prediction enhances the accuracy of orientation estimation, and mitigates the problem stated above. With the proposed method, a backbone achieves similar state-of-the-art orientation estimation performance to existing approaches with well-designed network structures.