pwc-archive/papers-with-abstracts huggingface dataset:
Unbiased Scene Graph Generation via Rich and Fair Semantic Extraction
Authors: Bin Wen, Jie Luo, Xianglong Liu, Lei Huang
Date: 2020-02-01 00:00:00
Tasks: Graph Generation, Relation, Scene Graph Generation, Unbiased Scene Graph Generation

Extracting graph representation of visual scenes in image is a challenging task in computer vision. Although there has been encouraging progress of scene graph generation in the past decade, we surprisingly find that the performance of existing approaches is largely limited by the strong biases, which mainly stem from (1) unconsciously assuming relations with certain semantic properties such as symmetric and (2) imbalanced annotations over different relations. To alleviate the negative effects of these biases, we proposed a new and simple architecture named Rich and Fair semantic extraction network (RiFa for short), to not only capture rich semantic properties of the relations, but also fairly predict relations with different scale of annotations. Using pseudo-siamese networks, RiFa embeds the subject and object respectively to distinguish their semantic differences and meanwhile preserve their underlying semantic properties. Then, it further predicts subject-object relations based on both the visual and semantic features of entities under certain contextual area, and fairly ranks the relation predictions for those with a few annotations. Experiments on the popular Visual Genome dataset show that RiFa achieves state-of-the-art performance under several challenging settings of scene graph task. Especially, it performs significantly better on capturing different semantic properties of relations, and obtains the best overall per relation performance.
AdvJND: Generating Adversarial Examples with Just Noticeable Difference
Authors: Zifei Zhang, Kai Qiao, Lingyun Jiang, Linyuan Wang, Bin Yan
Date: 2020-02-01 00:00:00
Tasks: image-classification, Image Classification

Compared with traditional machine learning models, deep neural networks perform better, especially in image classification tasks. However, they are vulnerable to adversarial examples. Adding small perturbations on examples causes a good-performance model to misclassify the crafted examples, without category differences in the human eyes, and fools deep models successfully. There are two requirements for generating adversarial examples: the attack success rate and image fidelity metrics. Generally, perturbations are increased to ensure the adversarial examples' high attack success rate; however, the adversarial examples obtained have poor concealment. To alleviate the tradeoff between the attack success rate and image fidelity, we propose a method named AdvJND, adding visual model coefficients, just noticeable difference coefficients, in the constraint of a distortion function when generating adversarial examples. In fact, the visual subjective feeling of the human eyes is added as a priori information, which decides the distribution of perturbations, to improve the image quality of adversarial examples. We tested our method on the FashionMNIST, CIFAR10, and MiniImageNet datasets. Adversarial examples generated by our AdvJND algorithm yield gradient distributions that are similar to those of the original inputs. Hence, the crafted noise can be hidden in the original inputs, thus improving the attack concealment significantly.
Deeply Activated Salient Region for Instance Search
Authors: Hui-Chu Xiao, Wan-Lei Zhao, Jie Lin, Chong-Wah Ngo
Date: 2020-02-01 00:00:00
Tasks: Image Retrieval, Instance Search

The performance of instance search depends heavily on the ability to locate and describe a wide variety of object instances in a video/image collection. Due to the lack of proper mechanism in locating instances and deriving feature representation, instance search is generally only effective for retrieving instances of known object categories. In this paper, a simple but effective instance-level feature representation is presented. Different from other approaches, the issues in class-agnostic instance localization and distinctive feature representation are considered. The former is achieved by detecting salient instance regions from an image by a layer-wise back-propagation process. The back-propagation starts from the last convolution layer of a pre-trained CNN that is originally used for classification. The back-propagation proceeds layer-by-layer until it reaches the input layer. This allows the salient instance regions in the input image from both known and unknown categories to be activated. Each activated salient region covers the full or more usually a major range of an instance. The distinctive feature representation is produced by average-pooling on the feature map of certain layer with the detected instance region. Experiments show that such kind of feature representation demonstrates considerably better performance over most of the existing approaches. In addition, we show that the proposed feature descriptor is also suitable for content-based image search.
Large Hole Image Inpainting With Compress-Decompression Network
Authors: Zhenghang Wu, Yidong Cui
Date: 2020-02-01 00:00:00
Tasks: Decoder, Image Inpainting, Super-Resolution, valid

Image inpainting technology can patch images with missing pixels. Existing methods propose convolutional neural networks to repair corrupted images. The networks focus on the valid pixels around the missing pixels, use the encoder-decoder structure to extract valuable information, and use the information to fix the vacancy. However, if the missing part is too large to provide useful information, the result will exist blur, color mixing, and object confusion. In order to patch the large hole image, we study the existing approaches and propose a new network, the compression-decompression network. The compression network takes responsibility for inpainting and generating a down-sample image. The decompression network takes responsibility for extending the down-sample image into the original resolution. We construct the compression network with the residual network and propose a similar texture selection algorithm to extend the image that is better than using the super-resolution network. We evaluate our model over Places2 and CelebA data set and use the similarity ratio as the metric. The result shows that our model has better performance when the inpainting task has many conflicts.
Domain segmentation and adjustment for generalized zero-shot learning
Authors: Xinsheng Wang, Shanmin Pang, Jihua Zhu
Date: 2020-02-01 00:00:00
Tasks: Generalized Zero-Shot Learning, Zero-Shot Learning

In the generalized zero-shot learning, synthesizing unseen data with generative models has been the most popular method to address the imbalance of training data between seen and unseen classes. However, this method requires that the unseen semantic information is available during the training stage, and training generative models is not trivial. Given that the generator of these models can only be trained with seen classes, we argue that synthesizing unseen data may not be an ideal approach for addressing the domain shift caused by the imbalance of the training data. In this paper, we propose to realize the generalized zero-shot recognition in different domains. Thus, unseen (seen) classes can avoid the effect of the seen (unseen) classes. In practice, we propose a threshold and probabilistic distribution joint method to segment the testing instances into seen, unseen and uncertain domains. Moreover, the uncertain domain is further adjusted to alleviate the domain shift. Extensive experiments on five benchmark datasets show that the proposed method exhibits competitive performance compared with that based on generative models.
Rigidity Properties of the Blum Medial Axis
Authors: James Damon
Date: 2020-02-01 00:00:00

We consider the Blum medial axis of a region in $\mathbb R^n$ with piecewise smooth boundary and examine its "rigidity properties", by which we mean properties preserved under diffeomorphisms of the regions preserving the medial axis. There are several possible versions of rigidity depending on what features of the Blum medial axis we wish to retain. We use a form of the cross ratio from projective geometry to show that in the case of four smooth sheets of the medial axis meeting along a branching submanifold, the cross ratio defines a function on the branching sheet which must be preserved under any diffeomorphism of the medial axis with another. Second, we show in the generic case, along a Y-branching submanifold that there are three cross ratios involving the three limiting tangent planes of the three smooth sheets and each of the hyperplanes defined by one of the radial lines and the tangent space to the Y-branching submanifold at the point, which again must be preserved. Moreover, the triple of cross ratios then locally uniquely determines the angles between the smooth sheets. Third, we observe that for a diffeomorphism of the region preserving the Blum medial axis and the infinitesimal directions of the radial lines, the second derivative of the diffeomorphism at points of the medial axis must satisfy a condition relating the radial shape operators and hence the differential geometry of the boundaries at corresponding boundary points.
Foreground object segmentation in RGB-D data implemented on GPU
Authors: Piotr Janus, Tomasz Kryjak, Marek Gorgon
Date: 2020-02-01 00:00:00
Tasks: GPU, Segmentation, Semantic Segmentation

This paper presents a GPU implementation of two foreground object segmentation algorithms: Gaussian Mixture Model (GMM) and Pixel Based Adaptive Segmenter (PBAS) modified for RGB-D data support. The simultaneous use of colour (RGB) and depth (D) data allows to improve segmentation accuracy, especially in case of colour camouflage, illumination changes and occurrence of shadows. Three GPUs were used to accelerate calculations: embedded NVIDIA Jetson TX2 (Maxwell architecture), mobile NVIDIA GeForce GTX 1050m (Pascal architecture) and efficient NVIDIA RTX 2070 (Turing architecture). Segmentation accuracy comparable to previously published works was obtained. Moreover, the use of a GPU platform allowed to get real-time image processing. In addition, the system has been adapted to work with two RGB-D sensors: RealSense D415 and D435 from Intel.
Few-Shot Scene Adaptive Crowd Counting Using Meta-Learning
Authors: Mahesh Kumar Krishna Reddy, Mohammad Hossain, Mrigank Rochan, Yang Wang
Date: 2020-02-01 00:00:00
Tasks: Crowd Counting, Meta-Learning

We consider the problem of few-shot scene adaptive crowd counting. Given a target camera scene, our goal is to adapt a model to this specific scene with only a few labeled images of that scene. The solution to this problem has potential applications in numerous real-world scenarios, where we ideally like to deploy a crowd counting model specially adapted to a target camera. We accomplish this challenge by taking inspiration from the recently introduced learning-to-learn paradigm in the context of few-shot regime. In training, our method learns the model parameters in a way that facilitates the fast adaptation to the target scene. At test time, given a target scene with a small number of labeled data, our method quickly adapts to that scene with a few gradient updates to the learned parameters. Our extensive experimental results show that the proposed approach outperforms other alternatives in few-shot scene adaptive crowd counting. Code is available at https://github.com/maheshkkumar/fscc.
Adversarial Generation of Continuous Implicit Shape Representations
Authors: Marian Kleineberg, Matthias Fey, Frank Weichert
Date: 2020-02-02 00:00:00

This work presents a generative adversarial architecture for generating three-dimensional shapes based on signed distance representations. While the deep generation of shapes has been mostly tackled by voxel and surface point cloud approaches, our generator learns to approximate the signed distance for any point in space given prior latent information. Although structurally similar to generative point cloud approaches, this formulation can be evaluated with arbitrary point density during inference, leading to fine-grained details in generated outputs. Furthermore, we study the effects of using either progressively growing voxel- or point-processing networks as discriminators, and propose a refinement scheme to strengthen the generator's capabilities in modeling the zero iso-surface decision boundary of shapes. We train our approach on the ShapeNet benchmark dataset and validate, both quantitatively and qualitatively, its performance in generating realistic 3D shapes.
Interpreting video features: a comparison of 3D convolutional networks and convolutional LSTM networks
Authors: Joonatan Mänttäri, Sofia Broomé, John Folkesson, Hedvig Kjellström
Date: 2020-02-02 00:00:00
Tasks: General Classification

A number of techniques for interpretability have been presented for deep learning in computer vision, typically with the goal of understanding what the networks have based their classification on. However, interpretability for deep video architectures is still in its infancy and we do not yet have a clear concept of how to decode spatiotemporal features. In this paper, we present a study comparing how 3D convolutional networks and convolutional LSTM networks learn features across temporally dependent frames. This is the first comparison of two video models that both convolve to learn spatial features but have principally different methods of modeling time. Additionally, we extend the concept of meaningful perturbation introduced by \cite{MeaningFulPert} to the temporal dimension, to identify the temporal part of a sequence most meaningful to the network for a classification decision. Our findings indicate that the 3D convolutional model concentrates on shorter events in the input sequence, and places its spatial focus on fewer, contiguous areas.
A Novel Graph based Trajectory Predictor with Pseudo Oracle
Authors: Biao Yang, Guocheng Yan, Pin Wang, Ching-Yao Chan, Xiang Song, Yang Chen
Date: 2020-02-02 00:00:00
Tasks: Graph Attention, Pedestrian Trajectory Prediction, Self-Driving Cars, Trajectory Prediction

Pedestrian trajectory prediction in dynamic scenes remains a challenging and critical problem in numerous applications, such as self-driving cars and socially aware robots. Challenges concentrate on capturing pedestrians' motion patterns and social interactions, as well as handling the future uncertainties. Recent studies focus on modeling pedestrians' motion patterns with recurrent neural networks, capturing social interactions with pooling-based or graph-based methods, and handling future uncertainties by using random Gaussian noise as the latent variable. However, they do not integrate specific obstacle avoidance experience (OAE) that may improve prediction performance. For example, pedestrians' future trajectories are always influenced by others in front. Here we propose GTPPO (Graph-based Trajectory Predictor with Pseudo Oracle), an encoder-decoder-based method conditioned on pedestrians' future behaviors. Pedestrians' motion patterns are encoded with a long short-term memory unit, which introduces the temporal attention to highlight specific time steps. Their interactions are captured by a graph-based attention mechanism, which draws OAE into the data-driven learning process of graph attention. Future uncertainties are handled by generating multi-modal outputs with an informative latent variable. Such a variable is generated by a novel pseudo oracle predictor, which minimizes the knowledge gap between historical and ground-truth trajectories. Finally, the GTPPO is evaluated on ETH, UCY and Stanford Drone datasets, and the results demonstrate state-of-the-art performance. Besides, the qualitative evaluations show successful cases of handling sudden motion changes in the future. Such findings indicate that GTPPO can peek into the future.
3D Shape Segmentation with Geometric Deep Learning
Authors: Davide Boscaini, Fabio Poiesi
Date: 2020-02-02 00:00:00
Tasks: Deep Learning, Segmentation, Semantic Segmentation

The semantic segmentation of 3D shapes with a high-density of vertices could be impractical due to large memory requirements. To make this problem computationally tractable, we propose a neural-network based approach that produces 3D augmented views of the 3D shape to solve the whole segmentation as sub-segmentation problems. 3D augmented views are obtained by projecting vertices and normals of a 3D shape onto 2D regular grids taken from different viewpoints around the shape. These 3D views are then processed by a Convolutional Neural Network to produce a probability distribution function (pdf) over the set of the semantic classes for each vertex. These pdfs are then re-projected on the original 3D shape and postprocessed using contextual information through Conditional Random Fields. We validate our approach using 3D shapes of publicly available datasets and of real objects that are reconstructed using photogrammetry techniques. We compare our approach against state-of-the-art alternatives.
Regularizing Reasons for Outfit Evaluation with Gradient Penalty
Authors: Xingxing Zou, Zhizhong Li, Ke Bai, Dahua Lin, Waikeung Wong
Date: 2020-02-02 00:00:00
Tasks: Sentence

In this paper, we build an outfit evaluation system which provides feedbacks consisting of a judgment with a convincing explanation. The system is trained in a supervised manner which faithfully follows the domain knowledge in fashion. We create the EVALUATION3 dataset which is annotated with judgment, the decisive reason for the judgment, and all corresponding attributes (e.g. print, silhouette, and material \etc.). In the training process, features of all attributes in an outfit are first extracted and then concatenated as the input for the intra-factor compatibility net. Then, the inter-factor compatibility net is used to compute the loss for judgment. We penalize the gradient of judgment loss of so that our Grad-CAM-like reason is regularized to be consistent with the labeled reason. In inference, according to the obtained information of judgment, reason, and attributes, a user-friendly explanation sentence is generated by the pre-defined templates. The experimental results show that the obtained network combines the advantages of high precision and good interpretation.
Effect of Analysis Window and Feature Selection on Classification of Hand Movements Using EMG Signal
Authors: Asad Ullah, Sarwan Ali, Imdadullah Khan, Muhammad Asad Khan, Safiullah Faizullah
Date: 2020-02-02 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Electromyography (EMG), feature selection, General Classification

Electromyography (EMG) signals have been successfully employed for driving prosthetic limbs of a single or double degree of freedom. This principle works by using the amplitude of the EMG signals to decide between one or two simpler movements. This method underperforms as compare to the contemporary advances done at the mechanical, electronics, and robotics end, and it lacks intuition. Recently, research on myoelectric control based on pattern recognition (PR) shows promising results with the aid of machine learning classifiers. Using the approach termed as, EMG-PR, EMG signals are divided into analysis windows, and features are extracted for each window. These features are then fed to the machine learning classifiers as input. By offering multiple class movements and intuitive control, this method has the potential to power an amputated subject to perform everyday life movements. In this paper, we investigate the effect of the analysis window and feature selection on classification accuracy of different hand and wrist movements using time-domain features. We show that effective data preprocessing and optimum feature selection helps to improve the classification accuracy of hand movements. We use publicly available hand and wrist gesture dataset of $40$ intact subjects for experimentation. Results computed using different classification algorithms show that the proposed preprocessing and features selection outperforms the baseline and achieve up to $98\%$ classification accuracy.
Neural Sign Language Translation by Learning Tokenization
Authors: Alptekin Orbay, Lale Akarun
Date: 2020-02-02 00:00:00
Tasks: Sign Language Translation, Transfer Learning, Translation

Sign Language Translation has attained considerable success recently, raising hopes for improved communication with the Deaf. A pre-processing step called tokenization improves the success of translations. Tokens can be learned from sign videos if supervised data is available. However, data annotation at the gloss level is costly, and annotated data is scarce. The paper utilizes Adversarial, Multitask, Transfer Learning to search for semi-supervised tokenization approaches without burden of additional labeling. It provides extensive experiments to compare all the methods in different settings to conduct a deeper analysis. In the case of no additional target annotation besides sentences, the proposed methodology attains 13.25 BLUE-4 and 36.28 ROUGE scores which improves the current state-of-the-art by 4 points in BLUE-4 and 5 points in ROUGE.
Novelty Detection via Non-Adversarial Generative Network
Authors: Chengwei Chen, Wang Yuan, Yuan Xie, Yanyun Qu, Yiqing Tao, Haichuan Song, Lizhuang Ma
Date: 2020-02-03 00:00:00
Tasks: Decoder, Image Reconstruction, Novelty Detection

One-class novelty detection is the process of determining if a query example differs from the training examples (the target class). Most of previous strategies attempt to learn the real characteristics of target sample by using generative adversarial networks (GANs) methods. However, the training process of GANs remains challenging, suffering from instability issues such as mode collapse and vanishing gradients. In this paper, by adopting non-adversarial generative networks, a novel decoder-encoder framework is proposed for novelty detection task, insteading of classical encoder-decoder style. Under the non-adversarial framework, both latent space and image reconstruction space are jointly optimized, leading to a more stable training process with super fast convergence and lower training losses. During inference, inspired by cycleGAN, we design a new testing scheme to conduct image reconstruction, which is the reverse way of training sequence. Experiments show that our model has the clear superiority over cutting-edge novelty detectors and achieves the state-of-the-art results on the datasets.
Automatic Pruning for Quantized Neural Networks
Authors: Luis Guerra, Bohan Zhuang, Ian Reid, Tom Drummond
Date: 2020-02-03 00:00:00
Tasks: Bayesian Optimization, Quantization

Neural network quantization and pruning are two techniques commonly used to reduce the computational complexity and memory footprint of these models for deployment. However, most existing pruning strategies operate on full-precision and cannot be directly applied to discrete parameter distributions after quantization. In contrast, we study a combination of these two techniques to achieve further network compression. In particular, we propose an effective pruning strategy for selecting redundant low-precision filters. Furthermore, we leverage Bayesian optimization to efficiently determine the pruning ratio for each layer. We conduct extensive experiments on CIFAR-10 and ImageNet with various architectures and precisions. In particular, for ResNet-18 on ImageNet, we prune 26.12% of the model size with Binarized Neural Network quantization, achieving a top-1 classification accuracy of 47.32% in a model of 2.47 MB and 59.30% with a 2-bit DoReFa-Net in 4.36 MB.