pwc-archive/papers-with-abstracts huggingface dataset:
Hybrid Trilinear and Bilinear Programming for Aligning Partially Overlapping Point Sets
Authors: Wei Lian, WangMeng Zuo
Date: 2021-01-19 00:00:00

In many applications, we need algorithms which can align partially overlapping point sets and are invariant to the corresponding transformations. In this work, a method possessing such properties is realized by minimizing the objective of the robust point matching (RPM) algorithm. We first show that the RPM objective is a cubic polynomial. We then utilize the convex envelopes of trilinear and bilinear monomials to derive its lower bound function. The resulting lower bound problem has the merit that it can be efficiently solved via linear assignment and low dimensional convex quadratic programming. We next develop a branch-and-bound (BnB) algorithm which only branches over the transformation variables and runs efficiently. Experimental results demonstrated better robustness of the proposed method against non-rigid deformation, positional noise and outliers in case that outliers are not mixed with inliers when compared with the state-of-the-art approaches. They also showed that it has competitive efficiency and scales well with problem size.
Deep Reinforcement Learning for Producing Furniture Layout in Indoor Scenes
Authors: Xinhan Di, Pengqian Yu
Date: 2021-01-19 00:00:00
Tasks: Deep Reinforcement Learning, Position, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In the industrial interior design process, professional designers plan the size and position of furniture in a room to achieve a satisfactory design for selling. In this paper, we explore the interior scene design task as a Markov decision process (MDP), which is solved by deep reinforcement learning. The goal is to produce an accurate position and size of the furniture simultaneously for the indoor layout task. In particular, we first formulate the furniture layout task as a MDP problem by defining the state, action, and reward function. We then design the simulated environment and train reinforcement learning agents to produce the optimal layout for the MDP formulation. We conduct our experiments on a large-scale real-world interior layout dataset that contains industrial designs from professional designers. Our numerical results demonstrate that the proposed model yields higher-quality layouts as compared with the state-of-art model. The developed simulator and codes are available at \url{https://github.com/CODE-SUBMIT/simulator1}.
Unsupervised Deep Learning for Handwritten Page Segmentation
Authors: Ahmad Droby, Berat Kurar Barakat, Borak Madi, Reem Alaasam, Jihad El-Sana
Date: 2021-01-19 00:00:00
Tasks: Deep Learning, Segmentation

Segmenting handwritten document images into regions with homogeneous patterns is an important pre-processing step for many document images analysis tasks. Hand-labeling data to train a deep learning model for layout analysis requires significant human effort. In this paper, we present an unsupervised deep learning method for page segmentation, which revokes the need for annotated images. A siamese neural network is trained to differentiate between patches using their measurable properties such as number of foreground pixels, and average component height and width. The network is trained that spatially nearby patches are similar. The network's learned features are used for page segmentation, where patches are classified as main and side text based on the extracted features. We tested the method on a dataset of handwritten document images with quite complex layouts. Our experiments show that the proposed unsupervised method is as effective as typical supervised methods.
Attention-Guided Black-box Adversarial Attacks with Large-Scale Multiobjective Evolutionary Optimization
Authors: Jie Wang, Zhaoxia Yin, Jing Jiang, Yang Du
Date: 2021-01-19 00:00:00
Proceeding: ICML Workshop AML 2021 7
Tasks: Adversarial Attack

Fooling deep neural networks (DNNs) with the black-box optimization has become a popular adversarial attack fashion, as the structural prior knowledge of DNNs is always unknown. Nevertheless, recent black-box adversarial attacks may struggle to balance their attack ability and visual quality of the generated adversarial examples (AEs) in tackling high-resolution images. In this paper, we propose an attention-guided black-box adversarial attack based on the large-scale multiobjective evolutionary optimization, termed as LMOA. By considering the spatial semantic information of images, we firstly take advantage of the attention map to determine the perturbed pixels. Instead of attacking the entire image, reducing the perturbed pixels with the attention mechanism can help to avoid the notorious curse of dimensionality and thereby improves the performance of attacking. Secondly, a large-scale multiobjective evolutionary algorithm is employed to traverse the reduced pixels in the salient region. Benefiting from its characteristics, the generated AEs have the potential to fool target DNNs while being imperceptible by the human vision. Extensive experimental results have verified the effectiveness of the proposed LMOA on the ImageNet dataset. More importantly, it is more competitive to generate high-resolution AEs with better visual quality compared with the existing black-box adversarial attacks.
BANet: Blur-aware Attention Networks for Dynamic Scene Deblurring
Authors: Fu-Jen Tsai, Yan-Tsung Peng, Yen-Yu Lin, Chung-Chi Tsai, Chia-Wen Lin
Date: 2021-01-19 00:00:00
Tasks: Deblurring, Image Deblurring, Image Restoration

Image motion blur results from a combination of object motions and camera shakes, and such blurring effect is generally directional and non-uniform. Previous research attempted to solve non-uniform blurs using self-recurrent multiscale, multi-patch, or multi-temporal architectures with self-attention to obtain decent results. However, using self-recurrent frameworks typically lead to a longer inference time, while inter-pixel or inter-channel self-attention may cause excessive memory usage. This paper proposes a Blur-aware Attention Network (BANet), that accomplishes accurate and efficient deblurring via a single forward pass. Our BANet utilizes region-based self-attention with multi-kernel strip pooling to disentangle blur patterns of different magnitudes and orientations and cascaded parallel dilated convolution to aggregate multi-scale content features. Extensive experimental results on the GoPro and RealBlur benchmarks demonstrate that the proposed BANet performs favorably against the state-of-the-arts in blurred image restoration and can provide deblurred results in real-time.
PICA: A Pixel Correlation-based Attentional Black-box Adversarial Attack
Authors: Jie Wang, Zhaoxia Yin, Jin Tang, Jing Jiang, Bin Luo
Date: 2021-01-19 00:00:00
Tasks: Adversarial Attack

The studies on black-box adversarial attacks have become increasingly prevalent due to the intractable acquisition of the structural knowledge of deep neural networks (DNNs). However, the performance of emerging attacks is negatively impacted when fooling DNNs tailored for high-resolution images. One of the explanations is that these methods usually focus on attacking the entire image, regardless of its spatial semantic information, and thereby encounter the notorious curse of dimensionality. To this end, we propose a pixel correlation-based attentional black-box adversarial attack, termed as PICA. Firstly, we take only one of every two neighboring pixels in the salient region as the target by leveraging the attentional mechanism and pixel correlation of images, such that the dimension of the black-box attack reduces. After that, a general multiobjective evolutionary algorithm is employed to traverse the reduced pixels and generate perturbations that are imperceptible by the human vision. Extensive experimental results have verified the effectiveness of the proposed PICA on the ImageNet dataset. More importantly, PICA is computationally more efficient to generate high-resolution adversarial examples compared with the existing black-box attacks.
VML-MOC: Segmenting a multiply oriented and curved handwritten text lines dataset
Authors: Berat Kurar Barakat, Rafi Cohen, Irina Rabaev, Jihad El-Sana
Date: 2021-01-19 00:00:00

This paper publishes a natural and very complicated dataset of handwritten documents with multiply oriented and curved text lines, namely VML-MOC dataset. These text lines were written as remarks on the page margins by different writers over the years. They appear at different locations within the orientations that range between 0 and 180 or as curvilinear forms. We evaluate a multi-oriented Gaussian based method to segment these handwritten text lines that are skewed or curved in any orientation. It achieves a mean pixel Intersection over Union score of 80.96% on the test documents. The results are compared with the results of a single-oriented Gaussian based text line segmentation method.
Object Tracking by Detection with Visual and Motion Cues
Authors: Niels Ole Salscheider
Date: 2021-01-19 00:00:00
Tasks: Autonomous Vehicles, Object Tracking, Optical Flow Estimation, Self-Driving Cars

Self-driving cars and other autonomous vehicles need to detect and track objects in camera images. We present a simple online tracking algorithm that is based on a constant velocity motion model with a Kalman filter, and an assignment heuristic. The assignment heuristic relies on four metrics: An embedding vector that describes the appearance of objects and can be used to re-identify them, a displacement vector that describes the object movement between two consecutive video frames, the Mahalanobis distance between the Kalman filter states and the new detections, and a class distance. These metrics are combined with a linear SVM, and then the assignment problem is solved by the Hungarian algorithm. We also propose an efficient CNN architecture that estimates these metrics. Our multi-frame model accepts two consecutive video frames which are processed individually in the backbone, and then optical flow is estimated on the resulting feature maps. This allows the network heads to estimate the displacement vectors. We evaluate our approach on the challenging BDD100K tracking dataset. Our multi-frame model achieves a good MOTA value of 39.1% with low localization error of 0.206 in MOTP. Our fast single-frame model achieves an even lower localization error of 0.202 in MOTP, and a MOTA value of 36.8%.
JigsawGAN: Auxiliary Learning for Solving Jigsaw Puzzles with Generative Adversarial Networks
Authors: Ru Li, Shuaicheng Liu, Guangfu Wang, Guanghui Liu, Bing Zeng
Date: 2021-01-19 00:00:00
Tasks: Auxiliary Learning, General Classification, Generative Adversarial Network, Self-Supervised Learning

The paper proposes a solution based on Generative Adversarial Network (GAN) for solving jigsaw puzzles. The problem assumes that an image is divided into equal square pieces, and asks to recover the image according to information provided by the pieces. Conventional jigsaw puzzle solvers often determine the relationships based on the boundaries of pieces, which ignore the important semantic information. In this paper, we propose JigsawGAN, a GAN-based auxiliary learning method for solving jigsaw puzzles with unpaired images (with no prior knowledge of the initial images). We design a multi-task pipeline that includes, (1) a classification branch to classify jigsaw permutations, and (2) a GAN branch to recover features to images in correct orders. The classification branch is constrained by the pseudo-labels generated according to the shuffled pieces. The GAN branch concentrates on the image semantic information, where the generator produces the natural images to fool the discriminator, while the discriminator distinguishes whether a given image belongs to the synthesized or the real target domain. These two branches are connected by a flow-based warp module that is applied to warp features to correct the order according to the classification results. The proposed method can solve jigsaw puzzles more efficiently by utilizing both semantic information and boundary information simultaneously. Qualitative and quantitative comparisons against several representative jigsaw puzzle solvers demonstrate the superiority of our method.
An Improvement of Object Detection Performance using Multi-step Machine Learnings
Authors: Tomoe Kishimoto, MasaHiko Saito, Junichi Tanaka, Yutaro Iiyama, Ryu Sawada, Koji Terashi
Date: 2021-01-19 00:00:00
Tasks: BIG-bench Machine Learning, Object, object-detection, Object Detection

Connecting multiple machine learning models into a pipeline is effective for handling complex problems. By breaking down the problem into steps, each tackled by a specific component model of the pipeline, the overall solution can be made accurate and explainable. This paper describes an enhancement of object detection based on this multi-step concept, where a post-processing step called the calibration model is introduced. The calibration model consists of a convolutional neural network, and utilizes rich contextual information based on the domain knowledge of the input. Improvements of object detection performance by 0.8-1.9 in average precision metric over existing object detectors have been observed using the new model.
Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets
Authors: Ke Li, Dengxin Dai, Ender Konukoglu, Luc van Gool
Date: 2021-01-19 00:00:00
Tasks: Data Augmentation, Hyperspectral Image Super-Resolution, Image Super-Resolution, Memorization, Super-Resolution

This work studies Hyperspectral image (HSI) super-resolution (SR). HSI SR is characterized by high-dimensional data and a limited amount of training examples. This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples. This work addresses these issues with three contributions. First, we observe that HSI SR and RGB image SR are correlated and develop a novel multi-tasking network to train them jointly so that the auxiliary task RGB image SR can provide additional supervision. Second, we propose a simple, yet effective data augmentation routine, termed Spectral Mixup, to construct effective virtual training samples to enlarge the training set. Finally, we extend the network to a semi-supervised setting so that it can learn from datasets containing only low-resolution HSIs. With these contributions, our method is able to learn from heterogeneous datasets and lift the requirement for having a large amount of HD HSI training samples. Extensive experiments on four standard datasets show that our method outperforms existing methods significantly and underpin the relevance of our contributions. Code has been made available at https://github.com/kli8996/HSISR.
Human Action Recognition Based on Multi-scale Feature Maps from Depth Video Sequences
Authors: Chang Li, Qian Huang, Xing Li, Qianhan Wu
Date: 2021-01-19 00:00:00
Tasks: Action Classification, Action Recognition, Temporal Action Localization

Human action recognition is an active research area in computer vision. Although great process has been made, previous methods mostly recognize actions based on depth data at only one scale, and thus they often neglect multi-scale features that provide additional information action recognition in practical application scenarios. In this paper, we present a novel framework focusing on multi-scale motion information to recognize human actions from depth video sequences. We propose a multi-scale feature map called Laplacian pyramid depth motion images(LP-DMI). We employ depth motion images (DMI) as the templates to generate the multi-scale static representation of actions. Then, we caculate LP-DMI to enhance multi-scale dynamic information of motions and reduces redundant static information in human bodies. We further extract the multi-granularity descriptor called LP-DMI-HOG to provide more discriminative features. Finally, we utilize extreme learning machine (ELM) for action classification. The proposed method yeilds the recognition accuracy of 93.41%, 85.12%, 91.94% on public MSRAction3D dataset, UTD-MHAD and DHA dataset. Through extensive experiments, we prove that our method outperforms state-of-the-art benchmarks.
Salient Object Detection via Integrity Learning
Authors: Mingchen Zhuge, Deng-Ping Fan, Nian Liu, Dingwen Zhang, Dong Xu, Ling Shao
Date: 2021-01-19 00:00:00
Tasks: Diversity, Object, object-detection, Object Detection, Salient Object Detection

Although current salient object detection (SOD) works have achieved significant progress, they are limited when it comes to the integrity of the predicted salient regions. We define the concept of integrity at both a micro and macro level. Specifically, at the micro level, the model should highlight all parts that belong to a certain salient object. Meanwhile, at the macro level, the model needs to discover all salient objects in a given image. To facilitate integrity learning for SOD, we design a novel Integrity Cognition Network (ICON), which explores three important components for learning strong integrity features. 1) Unlike existing models, which focus more on feature discriminability, we introduce a diverse feature aggregation (DFA) component to aggregate features with various receptive fields (i.e., kernel shape and context) and increase feature diversity. Such diversity is the foundation for mining the integral salient objects. 2) Based on the DFA features, we introduce an integrity channel enhancement (ICE) component with the goal of enhancing feature channels that highlight the integral salient objects, while suppressing the other distracting ones. 3) After extracting the enhanced features, the part-whole verification (PWV) method is employed to determine whether the part and whole object features have strong agreement. Such part-whole agreements can further improve the micro-level integrity for each salient object. To demonstrate the effectiveness of our ICON, comprehensive experiments are conducted on seven challenging benchmarks. Our ICON outperforms the baseline methods in terms of a wide range of metrics. Notably, our ICON achieves about 10% relative improvement over the previous best model in terms of average false negative ratio (FNR), on six datasets. Codes and results are available at: https://github.com/mczhuge/ICON.
Hyperspectral Image Denoising via Multi-modal and Double-weighted Tensor Nuclear Norm
Authors: Sheng Liu, Xiaozhen Xie, Wenfeng Kong
Date: 2021-01-19 00:00:00
Tasks: Denoising, Hyperspectral Image Denoising, Image Denoising, Image Restoration

Hyperspectral images (HSIs) usually suffer from different types of pollution. This severely reduces the quality of HSIs and limits the accuracy of subsequent processing tasks. HSI denoising can be modeled as a low-rank tensor denoising problem. Tensor nuclear norm (TNN) induced by tensor singular value decomposition plays an important role in this problem. In this letter, we first reconsider three inconspicuous but crucial phenomenons in TNN. In the Fourier transform domain of HSIs, different frequency slices (FS) contain different information; different singular values (SVs) of each FS also represent different information. The two physical phenomenons lie not only in the spectral mode but also in the spatial modes. Then based on them, we propose a multi-modal and double-weighted TNN. It can adaptively shrink the FS and SVs according to their physical meanings in all modes of HSIs. In the framework of the alternating direction method of multipliers, we design an effective alternating iterative strategy to optimize our proposed model. Denoised experiments on both synthetic and real HSI datasets demonstrate their superiority against related methods.
Deep Feedback Inverse Problem Solver
Authors: Wei-Chiu Ma, Shenlong Wang, Jiayuan Gu, Sivabalan Manivasagam, Antonio Torralba, Raquel Urtasun
Date: 2021-01-19 00:00:00
Proceeding: ECCV 2020 8
Tasks: Pose Estimation

We present an efficient, effective, and generic approach towards solving inverse problems. The key idea is to leverage the feedback signal provided by the forward process and learn an iterative update model. Specifically, at each iteration, the neural network takes the feedback as input and outputs an update on the current estimation. Our approach does not have any restrictions on the forward process; it does not require any prior knowledge either. Through the feedback information, our model not only can produce accurate estimations that are coherent to the input observation but also is capable of recovering from early incorrect predictions. We verify the performance of our approach over a wide range of inverse problems, including 6-DOF pose estimation, illumination estimation, as well as inverse kinematics. Comparing to traditional optimization-based methods, we can achieve comparable or better performance while being two to three orders of magnitude faster. Compared to deep learning-based approaches, our model consistently improves the performance on all metrics. Please refer to the project page for videos, animations, supplementary materials, etc.
Hyperdimensional computing as a framework for systematic aggregation of image descriptors
Authors: Peer Neubert, Stefan Schubert
Date: 2021-01-19 00:00:00
Proceeding: CVPR 2021 1

Image and video descriptors are an omnipresent tool in computer vision and its application fields like mobile robotics. Many hand-crafted and in particular learned image descriptors are numerical vectors with a potentially (very) large number of dimensions. Practical considerations like memory consumption or time for comparisons call for the creation of compact representations. In this paper, we use hyperdimensional computing (HDC) as an approach to systematically combine information from a set of vectors in a single vector of the same dimensionality. HDC is a known technique to perform symbolic processing with distributed representation in numerical vectors with thousands of dimensions. We present a HDC implementation that is suitable for processing the output of existing and future (deep-learning based) image descriptors. We discuss how this can be used as a framework to process descriptors together with additional knowledge by simple and fast vector operations. A concrete outcome is a novel HDC-based approach to aggregate a set of local image descriptors together with their image positions in a single holistic descriptor. The comparison to available holistic descriptors and aggregation methods on a series of standard mobile robotics place recognition experiments shows a 20% improvement in average performance compared to runner-up and 3.6x better worst-case performance.
Visualizing Missing Surfaces In Colonoscopy Videos using Shared Latent Space Representations
Authors: Shawn Mathew, Saad Nadeem, Arie Kaufman
Date: 2021-01-18 00:00:00
Tasks: Image-to-Image Translation, Translation

Optical colonoscopy (OC), the most prevalent colon cancer screening tool, has a high miss rate due to a number of factors, including the geometry of the colon (haustral fold and sharp bends occlusions), endoscopist inexperience or fatigue, endoscope field of view, etc. We present a framework to visualize the missed regions per-frame during the colonoscopy, and provides a workable clinical solution. Specifically, we make use of 3D reconstructed virtual colonoscopy (VC) data and the insight that VC and OC share the same underlying geometry but differ in color, texture and specular reflections, embedded in the OC domain. A lossy unpaired image-to-image translation model is introduced with enforced shared latent space for OC and VC. This shared latent space captures the geometric information while deferring the color, texture, and specular information creation to additional Gaussian noise input. This additional noise input can be utilized to generate one-to-many mappings from VC to OC and OC to OC. The code, data and trained models will be released via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.