pwc-archive/papers-with-abstracts huggingface dataset:
Extreme Relative Pose Network under Hybrid Representations
Authors: Zhenpei Yang, Siming Yan, Qi-Xing Huang
Date: 2019-12-25 00:00:00
Proceeding: CVPR 2020 6
Tasks: Pose Estimation, Translation

In this paper, we introduce a novel RGB-D based relative pose estimation approach that is suitable for small-overlapping or non-overlapping scans and can output multiple relative poses. Our method performs scene completion and matches the completed scans. However, instead of using a fixed representation for completion, the key idea is to utilize hybrid representations that combine 360-image, 2D image-based layout, and planar patches. This approach offers adaptively feature representations for relative pose estimation. Besides, we introduce a global-2-local matching procedure, which utilizes initial relative poses obtained during the global phase to detect and then integrate geometric relations for pose refinement. Experimental results justify the potential of this approach across a wide range of benchmark datasets. For example, on ScanNet, the rotation translation errors of the top-1/top-5 predictions of our approach are 28.6/0.90m and 16.8/0.76m, respectively. Our approach also considerably boosts the performance of multi-scan reconstruction in few-view reconstruction settings.
Thermostatic control for demand response using distributed averaging and deep neural networks
Authors: Kshitij Singh, Pratik K. Bajaria
Date: 2019-12-25 00:00:00
Tasks: Management

Smart buildings are the need of the day with increasing demand-supply ratios and deficiency to generate considerably. In any modern non-industrial infrastructure, these demands mainly comprise of thermostatically controlled loads (TCLs), which can be manoeuvred. TCL loads like air-conditioner, heater, refrigerator, are ubiquitous, and their operating times can be controlled to achieve desired aggregate power. This power aggregation, in turn, helps achieve load management targets and thereby serve as ancillary service (AS) to the power grid. In this work, a distributed averaging protocol is used to achieve the desired power aggregate set by the utility using steady-state desynchronization. The results are verified using a computer program for a homogeneous and heterogeneous population of TCLs. Further, load following scenario has been implemented using the utility as a reference. Apart from providing a significant AS to the power grid, the proposed idea also helps reduce the amplitude of power system oscillations imparted by the TCLs. Hardware-based results are obtained to verify its implementation feasibility in real-time. Additionally, we extend this idea to data-driven paradigm and provide comparisons therein.
Multi-Modal Attention-based Fusion Model for Semantic Segmentation of RGB-Depth Images
Authors: Fahimeh Fooladgar, Shohreh Kasaei
Date: 2019-12-25 00:00:00
Tasks: Scene Understanding, Segmentation, Semantic Segmentation

The 3D scene understanding is mainly considered as a crucial requirement in computer vision and robotics applications. One of the high-level tasks in 3D scene understanding is semantic segmentation of RGB-Depth images. With the availability of RGB-D cameras, it is desired to improve the accuracy of the scene understanding process by exploiting the depth features along with the appearance features. As depth images are independent of illumination, they can improve the quality of semantic labeling alongside RGB images. Consideration of both common and specific features of these two modalities improves the performance of semantic segmentation. One of the main problems in RGB-Depth semantic segmentation is how to fuse or combine these two modalities to achieve more advantages of each modality while being computationally efficient. Recently, the methods that encounter deep convolutional neural networks have reached the state-of-the-art results by early, late, and middle fusion strategies. In this paper, an efficient encoder-decoder model with the attention-based fusion block is proposed to integrate mutual influences between feature maps of these two modalities. This block explicitly extracts the interdependences among concatenated feature maps of these modalities to exploit more powerful feature maps from RGB-Depth images. The extensive experimental results on three main challenging datasets of NYU-V2, SUN RGB-D, and Stanford 2D-3D-Semantic show that the proposed network outperforms the state-of-the-art models with respect to computational cost as well as model size. Experimental results also illustrate the effectiveness of the proposed lightweight attention-based fusion model in terms of accuracy.
Unity in Diversity: Learning Distributed Heterogeneous Sentence Representation for Extractive Summarization
Authors: Abhishek Kumar Singh, Manish Gupta, Vasudeva Varma
Date: 2019-12-25 00:00:00
Tasks: Diversity, Extractive Summarization, Extractive Text Summarization, Natural Language Understanding, Sentence, Text Summarization, Unity

Automated multi-document extractive text summarization is a widely studied research problem in the field of natural language understanding. Such extractive mechanisms compute in some form the worthiness of a sentence to be included into the summary. While the conventional approaches rely on human crafted document-independent features to generate a summary, we develop a data-driven novel summary system called HNet, which exploits the various semantic and compositional aspects latent in a sentence to capture document independent features. The network learns sentence representation in a way that, salient sentences are closer in the vector space than non-salient sentences. This semantic and compositional feature vector is then concatenated with the document-dependent features for sentence ranking. Experiments on the DUC benchmark datasets (DUC-2001, DUC-2002 and DUC-2004) indicate that our model shows significant performance gain of around 1.5-2 points in terms of ROUGE score compared with the state-of-the-art baselines.
Look, Listen, and Act: Towards Audio-Visual Embodied Navigation
Authors: Chuang Gan, Yiwei Zhang, Jiajun Wu, Boqing Gong, Joshua B. Tenenbaum
Date: 2019-12-25 00:00:00
Tasks: Navigate

A crucial ability of mobile intelligent agents is to integrate the evidence from multiple sensory inputs in an environment and to make a sequence of actions to reach their goals. In this paper, we attempt to approach the problem of Audio-Visual Embodied Navigation, the task of planning the shortest path from a random starting location in a scene to the sound source in an indoor environment, given only raw egocentric visual and audio sensory data. To accomplish this task, the agent is required to learn from various modalities, i.e. relating the audio signal to the visual environment. Here we describe an approach to audio-visual embodied navigation that takes advantage of both visual and audio pieces of evidence. Our solution is based on three key ideas: a visual perception mapper module that constructs its spatial memory of the environment, a sound perception module that infers the relative location of the sound source from the agent, and a dynamic path planner that plans a sequence of actions based on the audio-visual observations and the spatial memory of the environment to navigate toward the goal. Experimental results on a newly collected Visual-Audio-Room dataset using the simulated multi-modal environment demonstrate the effectiveness of our approach over several competitive baselines.
Neural ODEs for Image Segmentation with Level Sets
Authors: Rafael Valle, Fitsum Reda, Mohammad Shoeybi, Patrick Legresley, Andrew Tao, Bryan Catanzaro
Date: 2019-12-25 00:00:00
Tasks: Deep Learning, Image Segmentation, object-detection, Object Detection, RGB Salient Object Detection, Salient Object Detection, Segmentation, Semantic Segmentation

We propose a novel approach for image segmentation that combines Neural Ordinary Differential Equations (NODEs) and the Level Set method. Our approach parametrizes the evolution of an initial contour with a NODE that implicitly learns from data a speed function describing the evolution. In addition, for cases where an initial contour is not available and to alleviate the need for careful choice or design of contour embedding functions, we propose a NODE-based method that evolves an image embedding into a dense per-pixel semantic label space. We evaluate our methods on kidney segmentation (KiTS19) and on salient object detection (PASCAL-S, ECSSD and HKU-IS). In addition to improving initial contours provided by deep learning models while using a fraction of their number of parameters, our approach achieves F scores that are higher than several state-of-the-art deep learning algorithms.
Deep Learning-based Vehicle Behaviour Prediction For Autonomous Driving Applications: A Review
Authors: Sajjad Mozaffari, Omar Y. Al-Jarrah, Mehrdad Dianati, Paul Jennings, Alexandros Mouzakitis
Date: 2019-12-25 00:00:00
Tasks: Autonomous Driving, Deep Learning, Prediction

Behaviour prediction function of an autonomous vehicle predicts the future states of the nearby vehicles based on the current and past observations of the surrounding environment. This helps enhance their awareness of the imminent hazards. However, conventional behaviour prediction solutions are applicable in simple driving scenarios that require short prediction horizons. Most recently, deep learning-based approaches have become popular due to their superior performance in more complex environments compared to the conventional approaches. Motivated by this increased popularity, we provide a comprehensive review of the state-of-the-art of deep learning-based approaches for vehicle behaviour prediction in this paper. We firstly give an overview of the generic problem of vehicle behaviour prediction and discuss its challenges, followed by classification and review of the most recent deep learning-based solutions based on three criteria: input representation, output type, and prediction method. The paper also discusses the performance of several well-known solutions, identifies the research gaps in the literature and outlines potential new research directions.
Learning Controllable Disentangled Representations with Decorrelation Regularization
Authors: Zengjie Song, Oluwasanmi Koyejo, Jiangshe Zhang
Date: 2019-12-25 00:00:00
Tasks: Decoder, Disentanglement

A crucial problem in learning disentangled image representations is controlling the degree of disentanglement during image editing, while preserving the identity of objects. In this work, we propose a simple yet effective model with the encoder-decoder architecture to address this challenge. To encourage disentanglement, we devise a distance covariance based decorrelation regularization. Further, for the reconstruction step, our model leverages a soft target representation combined with the latent image code. By exploiting the real-valued space of the soft target representations, we are able to synthesize novel images with the designated properties. We also design a classification based protocol to quantitatively evaluate the disentanglement strength of our model. Experimental results show that the proposed model competently disentangles factors of variation, and is able to manipulate face images to synthesize the desired attributes.
Learning to Answer Ambiguous Questions with Knowledge Graph
Authors: Yikai Zhu, Jianhao Shen, Ming Zhang
Date: 2019-12-25 00:00:00
Tasks: Question Answering

In the task of factoid question answering over knowledge base, many questions have more than one plausible interpretation. Previous works on SimpleQuestions assume only one interpretation as the ground truth for each question, so they lack the ability to answer ambiguous questions correctly. In this paper, we present a new way to utilize the dataset that takes into account the existence of ambiguous questions. Then we introduce a simple and effective model which combines local knowledge subgraph with attention mechanism. Our experimental results show that our approach achieves outstanding performance in this task.
Asymmetric GAN for Unpaired Image-to-image Translation
Authors: Yu Li, Sheng Tang, Rui Zhang, Yongdong Zhang, Jintao Li, Shuicheng Yan
Date: 2019-12-25 00:00:00
Tasks: Image-to-Image Translation, Translation

Unpaired image-to-image translation problem aims to model the mapping from one domain to another with unpaired training data. Current works like the well-acknowledged Cycle GAN provide a general solution for any two domains through modeling injective mappings with a symmetric structure. While in situations where two domains are asymmetric in complexity, i.e., the amount of information between two domains is different, these approaches pose problems of poor generation quality, mapping ambiguity, and model sensitivity. To address these issues, we propose Asymmetric GAN (AsymGAN) to adapt the asymmetric domains by introducing an auxiliary variable (aux) to learn the extra information for transferring from the information-poor domain to the information-rich domain, which improves the performance of state-of-the-art approaches in the following ways. First, aux better balances the information between two domains which benefits the quality of generation. Second, the imbalance of information commonly leads to mapping ambiguity, where we are able to model one-to-many mappings by tuning aux, and furthermore, our aux is controllable. Third, the training of Cycle GAN can easily make the generator pair sensitive to small disturbances and variations while our model decouples the ill-conditioned relevance of generators by injecting aux during training. We verify the effectiveness of our proposed method both qualitatively and quantitatively on asymmetric situation, label-photo task, on Cityscapes and Helen datasets, and show many applications of asymmetric image translations. In conclusion, our AsymGAN provides a better solution for unpaired image-to-image translation in asymmetric domains.
Improving Visual Recognition using Ambient Sound for Supervision
Authors: Rohan Mahadev, Hongyu Lu
Date: 2019-12-25 00:00:00

Our brains combine vision and hearing to create a more elaborate interpretation of the world. When the visual input is insufficient, a rich panoply of sounds can be used to describe our surroundings. Since more than 1,000 hours of videos are uploaded to the internet everyday, it is arduous, if not impossible, to manually annotate these videos. Therefore, incorporating audio along with visual data without annotations is crucial for leveraging this explosion of data for recognizing and understanding objects and scenes. Owens,et.al suggest that a rich representation of the physical world can be learned by using a convolutional neural network to predict sound textures associated with a given video frame. We attempt to reproduce the claims from their experiments, of which the code is not publicly available. In addition, we propose improvements in the pretext task that result in better performance in other downstream computer vision tasks.
DDI-100: Dataset for Text Detection and Recognition
Authors: Ilia Zharikov, Filipp Nikitin, Ilia Vasiliev, Vladimir Dokholyan
Date: 2019-12-25 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR), Text Detection

Nowadays document analysis and recognition remain challenging tasks. However, only a few datasets designed for text detection (TD) and optical character recognition (OCR) problems exist. In this paper we present Distorted Document Images dataset (DDI-100) and demonstrate its usefulness in a wide range of document analysis problems. DDI-100 dataset is a synthetic dataset based on 7000 real unique document pages and consists of more than 100000 augmented images. Ground truth comprises text and stamp masks, text and characters bounding boxes with relevant annotations. Validation of DDI-100 dataset was conducted using several TD and OCR models that show high-quality performance on real data.
Confounder Selection via Support Intersection
Authors: Shinyuu Lee, Yuru Zhu
Date: 2019-12-25 00:00:00
Tasks: Variable Selection

Confounding matters in almost all observational studies that focus on causality. In order to eliminate bias caused by connfounders, oftentimes a substantial number of features need to be collected in the analysis. In this case, large p small n problem can arise and dimensional reduction technique is required. However, the traditional variable selection methods which focus on prediction are problematic in this setting. Throughout this paper, we analyze this issue in detail and assume the sparsity of confounders which is different from the previous works. Under this assumption we propose several variable selection methods based on support intersection to pick out the confounders. Also we discussed the different approaches for estimation of causal effect and unconfoundedness test. To aid in our description, finally we provide numerical simulations to support our claims and compare to common heuristic methods, as well as applications on real dataset.
Extending Multi-Object Tracking systems to better exploit appearance and 3D information
Authors: Kanchana Ranasinghe, Sahan Liyanaarachchi, Harsha Ranasinghe, Mayuka Jayawardhana
Date: 2019-12-25 00:00:00
Tasks: Multi-Object Tracking, Object, Object Tracking, Real-Time Multi-Object Tracking

Tracking multiple objects in real time is essential for a variety of real-world applications, with self-driving industry being at the foremost. This work involves exploiting temporally varying appearance and motion information for tracking. Siamese networks have recently become highly successful at appearance based single object tracking and Recurrent Neural Networks have started dominating both motion and appearance based tracking. Our work focuses on combining Siamese networks and RNNs to exploit appearance and motion information respectively to build a joint system capable of real time multi-object tracking. We further explore heuristics based constraints for tracking in the Birds Eye View Space for efficiently exploiting 3D information as a constrained optimization problem for track prediction.
Competing Ratio Loss for Discriminative Multi-class Image Classification
Authors: Ke Zhang, Yurong Guo, Xinsheng Wang, Dongliang Chang, Zhenbing Zhao, Zhanyu Ma, Tony X. Han
Date: 2019-12-25 00:00:00
Tasks: Age Estimation, Classification, Fine-Grained Image Classification, General Classification, image-classification, Image Classification, Multi-class Classification

The development of deep convolutional neural network architecture is critical to the improvement of image classification task performance. Many image classification studies use deep convolutional neural network and focus on modifying the network structure to improve image classification performance. Conversely, our study focuses on loss function design. Cross-entropy Loss (CEL) has been widely used for training deep convolutional neural network for the task of multi-class classification. Although CEL has been successfully implemented in several image classification tasks, it only focuses on the posterior probability of the correct class. For this reason, a negative log likelihood ratio loss (NLLR) was proposed to better differentiate between the correct class and the competing incorrect ones. However, during the training of the deep convolutional neural network, the value of NLLR is not always positive or negative, which severely affects the convergence of NLLR. Our proposed competing ratio loss (CRL) calculates the posterior probability ratio between the correct class and the competing incorrect classes to further enlarge the probability difference between the correct and incorrect classes. We added hyperparameters to CRL, thereby ensuring its value to be positive and that the update size of backpropagation is suitable for the CRL's fast convergence. To demonstrate the performance of CRL, we conducted experiments on general image classification tasks (CIFAR10/100, SVHN, ImageNet), the fine-grained image classification tasks (CUB200-2011 and Stanford Car), and the challenging face age estimation task (using Adience). Experimental results show the effectiveness and robustness of the proposed loss function on different deep convolutional neural network architectures and different image classification tasks.
Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
Authors: Guangxiang Zhao, Junyang Lin, Zhiyuan Zhang, Xuancheng Ren, Qi Su, Xu sun
Date: 2019-12-25 00:00:00
Tasks: Image Captioning, Language Modeling, Language Modelling, Machine Translation, Translation

Self-attention based Transformer has demonstrated the state-of-the-art performances in a number of natural language processing tasks. Self-attention is able to model long-term dependencies, but it may suffer from the extraction of irrelevant information in the context. To tackle the problem, we propose a novel model called \textbf{Explicit Sparse Transformer}. Explicit Sparse Transformer is able to improve the concentration of attention on the global context through an explicit selection of the most relevant segments. Extensive experimental results on a series of natural language processing and computer vision tasks, including neural machine translation, image captioning, and language modeling, all demonstrate the advantages of Explicit Sparse Transformer in model performance. We also show that our proposed sparse attention method achieves comparable or better results than the previous sparse attention method, but significantly reduces training and testing time. For example, the inference speed is twice that of sparsemax in Transformer model. Code will be available at \url{https://github.com/lancopku/Explicit-Sparse-Transformer}
Ranking and Classification driven Feature Learning for Person Re_identification
Authors: Zhiguang Zhang
Date: 2019-12-25 00:00:00
Tasks: Classification, General Classification, Metric Learning, Person Re-Identification, Semantic Similarity, Semantic Textual Similarity, Triplet

Person re-identification has attracted many researchers' attention for its wide application, but it is still a very challenging task because only part of the image information can be used for personnel matching. Most of current methods uses CNN to learn to embeddings that can capture semantic similarity information among data points. Many of the state-of-the-arts methods use complex network structures with multiple branches that fuse multiple features while training or testing, using classification loss, Triplet loss or a combination of the two as loss function. However, the method that using Triplet loss as loss function converges slowly, and the method in which pull features of the same class as close as possible in features space leads to poor feature stability. This paper will combine the ranking motivated structured loss, proposed a new metric learning loss function that make the features of the same class are sparsely distributed into the range of small hyperspheres and the features of different classes are uniformly distributed at a clearly angle. And adopted a new single-branch network structure that only using global feature can also get great performance. The validity of our method is verified on the Market1501 and DukeMTMC-ReID person re-identification datasets. Finally acquires 90.9% rank-1 accuracy and 80.8% mAP on DukeMTMC-reID, 95.3% rank-1 accuracy and 88.7% mAP on Market1501. Codes and models are available in Github.https://github.com/Qidian213/Ranked_Person_ReID.