pwc-archive/papers-with-abstracts huggingface dataset:
Distributed and Consistent Multi-Image Feature Matching via QuickMatch
Authors: Zachary Serlin, Guang Yang, Brandon Sookraj, Calin Belta, Roberto Tron
Date: 2019-10-29 00:00:00
Tasks: Object, object-detection, Object Detection, Simultaneous Localization and Mapping

In this work we consider the multi-image object matching problem, extend a centralized solution of the problem to a distributed solution, and present an experimental application of the centralized solution. Multi-image feature matching is a keystone of many applications, including simultaneous localization and mapping, homography, object detection, and structure from motion. We first review the QuickMatch algorithm for multi-image feature matching. We then present a scheme for distributing sets of features across computational units (agents) that largely preserves feature match quality and minimizes communication between agents (avoiding, in particular, the need of flooding all data to all agents). Finally, we show how QuickMatch performs on an object matching test with low quality images. The centralized QuickMatch algorithm is compared to other standard matching algorithms, while the Distributed QuickMatch algorithm is compared to the centralized algorithm in terms of preservation of match consistency. The presented experiment shows that QuickMatch matches features across a large number of images and features in larger numbers and more accurately than standard techniques.
Resolution-independent meshes of super pixels
Authors: Vitaliy Kurlin, Philip Smith
Date: 2019-10-29 00:00:00
Tasks: Clustering, Segmentation, Superpixels

The over-segmentation into superpixels is an important preprocessing step to smartly compress the input size and speed up higher level tasks. A superpixel was traditionally considered as a small cluster of square-based pixels that have similar color intensities and are closely located to each other. In this discrete model the boundaries of superpixels often have irregular zigzags consisting of horizontal or vertical edges from a given pixel grid. However digital images represent a continuous world, hence the following continuous model in the resolution-independent formulation can be more suitable for the reconstruction problem. Instead of uniting squares in a grid, a resolution-independent superpixel is defined as a polygon that has straight edges with any possible slope at subpixel resolution. The harder continuous version of the over-segmentation problem is to split an image into polygons and find a best (say, constant) color of each polygon so that the resulting colored mesh well approximates the given image. Such a mesh of polygons can be rendered at any higher resolution with all edges kept straight. We propose a fast conversion of any traditional superpixels into polygons and guarantees that their straight edges do not intersect. The meshes based on the superpixels SEEDS (Superpixels Extracted via Energy-Driven Sampling) and SLIC (Simple Linear Iterative Clustering) are compared with past meshes based on the Line Segment Detector. The experiments on the Berkeley Segmentation Database confirm that the new superpixels have more compact shapes than pixel-based superpixels.
Deep Learning for Hyperspectral Image Classification: An Overview
Authors: Shutao Li, Weiwei Song, Leyuan Fang, Yushi Chen, Pedram Ghamisi, Jón Atli Benediktsson
Date: 2019-10-26 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Deep Learning, General Classification, Hyperspectral Image Classification, image-classification, Image Classification

Hyperspectral image (HSI) classification has become a hot topic in the field of remote sensing. In general, the complex characteristics of hyperspectral data make the accurate classification of such data challenging for traditional machine learning methods. In addition, hyperspectral imaging often deals with an inherently nonlinear relation between the captured spectral information and the corresponding materials. In recent years, deep learning has been recognized as a powerful feature-extraction tool to effectively address nonlinear problems and widely used in a number of image processing tasks. Motivated by those successful applications, deep learning has also been introduced to classify HSIs and demonstrated good performance. This survey paper presents a systematic review of deep learning-based HSI classification literatures and compares several strategies for this topic. Specifically, we first summarize the main challenges of HSI classification which cannot be effectively overcome by traditional machine learning methods, and also introduce the advantages of deep learning to handle these problems. Then, we build a framework which divides the corresponding works into spectral-feature networks, spatial-feature networks, and spectral-spatial-feature networks to systematically review the recent achievements in deep learning-based HSI classification. In addition, considering the fact that available training samples in the remote sensing field are usually very limited and training deep networks require a large number of samples, we include some strategies to improve classification performance, which can provide some guidelines for future studies on this topic. Finally, several representative deep learning-based classification methods are conducted on real HSIs in our experiments.
Deep Multi-Magnification Networks for Multi-Class Breast Cancer Image Segmentation
Authors: David Joon Ho, Dig V. K. Yarlagadda, Timothy M. D'Alfonso, Matthew G. Hanna, Anne Grabenstetter, Peter Ntiamoah, Edi Brogi, Lee K. Tan, Thomas J. Fuchs
Date: 2019-10-29 00:00:00
Tasks: Decoder, Image Segmentation, Semantic Segmentation, whole slide images

Pathologic analysis of surgical excision specimens for breast carcinoma is important to evaluate the completeness of surgical excision and has implications for future treatment. This analysis is performed manually by pathologists reviewing histologic slides prepared from formalin-fixed tissue. In this paper, we present Deep Multi-Magnification Network trained by partial annotation for automated multi-class tissue segmentation by a set of patches from multiple magnifications in digitized whole slide images. Our proposed architecture with multi-encoder, multi-decoder, and multi-concatenation outperforms other single and multi-magnification-based architectures by achieving the highest mean intersection-over-union, and can be used to facilitate pathologists' assessments of breast cancer.
A Robust Pavement Mapping System Based on Normal-Constrained Stereo Visual Odometry
Authors: Huaiyang Huang, Rui Fan, Yilong Zhu, Ming Liu, Ioannis Pitas
Date: 2019-10-29 00:00:00
Tasks: Visual Odometry

Pavement condition is crucial for civil infrastructure maintenance. This task usually requires efficient road damage localization, which can be accomplished by the visual odometry system embedded in unmanned aerial vehicles (UAVs). However, the state-of-the-art visual odometry and mapping methods suffer from large drift under the degeneration of the scene structure. To alleviate this issue, we integrate normal constraints into the visual odometry process, which greatly helps to avoid large drift. By parameterizing the normal vector on the tangential plane, the normal factors are coupled with traditional reprojection factors in the pose optimization procedure. The experimental results demonstrate the effectiveness of the proposed system. The overall absolute trajectory error is improved by approximately 20%, which indicates that the estimated trajectory is much more accurate than that obtained using other state-of-the-art methods.
Results from the Robocademy ITN: Autonomy, Disturbance Rejection and Perception for Advanced Marine Robotics
Authors: Matias Valdenegro-Toro, Mariela De Lucas Alvarez, Mariia Dmitrieva, Bilal Wehbe, Georgios Salavasidis, Shahab Heshmati-Alamdari, Juan F. Fuentes-Pérez, Veronika Yordanova, Klemen Istenič, Thomas Guerneve
Date: 2019-10-29 00:00:00

Marine and Underwater resources are important part of the economy of many countries. This requires significant financial resources into their construction and maintentance. Robotics is expected to fill this void, by automating and/or removing humans from hostile environments in order to easily perform maintenance tasks. The Robocademy Marie Sklodowska-Curie Initial Training Network was funded by the European Union's FP7 research program in order to train 13 Fellows into world-leading researchers in Marine and Underwater Robotics. The fellows developed guided research into three areas of key importance: Autonomy, Disturbance Rejection, and Perception. This paper presents a summary of the fellows' research in the three action lines. 71 scientific publications were the primary result of this project, with many other publications currently in the pipeline. Most of the fellows have found employment in Europe, which shows the high demand for this kind of experts. We believe the results from this project are already having an impact in the marine robotics industry, as key technologies are being adopted already.
On the Benefit of Adversarial Training for Monocular Depth Estimation
Authors: Rick Groenendijk, Sezer Karaoglu, Theo Gevers, Thomas Mensink
Date: 2019-10-29 00:00:00
Tasks: Depth Estimation, Depth Prediction, Generative Adversarial Network, Image Reconstruction, Monocular Depth Estimation

In this paper we address the benefit of adding adversarial training to the task of monocular depth estimation. A model can be trained in a self-supervised setting on stereo pairs of images, where depth (disparities) are an intermediate result in a right-to-left image reconstruction pipeline. For the quality of the image reconstruction and disparity prediction, a combination of different losses is used, including L1 image reconstruction losses and left-right disparity smoothness. These are local pixel-wise losses, while depth prediction requires global consistency. Therefore, we extend the self-supervised network to become a Generative Adversarial Network (GAN), by including a discriminator which should tell apart reconstructed (fake) images from real images. We evaluate Vanilla GANs, LSGANs and Wasserstein GANs in combination with different pixel-wise reconstruction losses. Based on extensive experimental evaluation, we conclude that adversarial training is beneficial if and only if the reconstruction loss is not too constrained. Even though adversarial training seems promising because it promotes global consistency, non-adversarial training outperforms (or is on par with) any method trained with a GAN when a constrained reconstruction loss is used in combination with batch normalisation. Based on the insights of our experimental evaluation we obtain state-of-the art monocular depth estimation results by using batch normalisation and different output scales.
Spatial Residual Layer and Dense Connection Block Enhanced Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition
Authors: Cong Wu, Xiao-Jun Wu, Josef Kittler
Date: 2019-10-27 00:00:00
Proceeding: IEEE International Conference on Computer Vision (ICCV), 2019 2019 10
Tasks: Action Recognition, Skeleton Based Action Recognition

Recent research has shown that modeling the dynamic joint features of the human body by a graph convolutional network (GCN) is a groundbreaking approach for skeleton-based action recognition, especially for the recognition of the body-motion, human-object and human-human interactions. Nevertheless, how to model and utilize coherent skeleton information comprehensively is still an open problem. In order to capture the rich spatiotemporal information and utilize features more effectively, we introduce a spatial residual layer and a dense connection block enhanced spatial temporal graph convolutional network. More specifically, our work introduces three aspects. Firstly, we extend spatial graph convolution to spatial temporal graph convolution of cross-domain residual to extract more precise and informative spatiotemporal feature, and reduce the training complexity by feature fusion in the, so-called, spatial residual layer. Secondly, instead of simply superimposing multiple similar layers, we use dense connection to take full advantage of the global information. Thirdly, we combine the above mentioned two components to create a spatial temporal graph convolutional network (ST-GCN), referred to as SDGCN. The proposed graph representation has a new structure. We perform extensive experiments on two large datasets: Kinetics and NTU-RGB+D. Our method achieves a great improvement in performance compared to the mainstream methods. We evaluate our method quantitatively and qualitatively, thus proving its effectiveness.
Stanford Neural Machine Translation Systems for Spoken Language Domains
Authors: Minh-Thang Luong, Christopher D. Manning
Date: 2015-12-08 00:00:00
Proceeding: IWSLT 2015 2015 12
Tasks: Machine Translation, NMT, Translation

Neural Machine Translation (NMT), though recently developed, has shown promising results for various language pairs. Despite that, NMT has only been applied to mostly formal texts such as those in the WMT shared tasks. This work further explores the effectiveness of NMT in spoken language domains by participating in the MT track of the IWSLT 2015. We consider two scenarios: (a) how to adapt existing NMT systems to a new domain and (b) the generalization of NMT to low-resource language pairs. Our results demonstrate that using an existing NMT framework1, we can achieve competitive results in the aforementioned scenarios when translating from English to German and Vietnamese. Notably, we have advanced state-of-the-art results in the IWSLT EnglishGerman MT track by up to 5.2 BLEU points.
ALFA: A Dataset for UAV Fault and Anomaly Detection
Authors: Azarakhsh Keipour, Mohammadreza Mousaei, Sebastian Scherer
Date: 2019-07-14 00:00:00
Tasks: Anomaly Detection, Fault Detection

We present a dataset of several fault types in control surfaces of a fixed-wing Unmanned Aerial Vehicle (UAV) for use in Fault Detection and Isolation (FDI) and Anomaly Detection (AD) research. Currently, the dataset includes processed data for 47 autonomous flights with scenarios for eight different types of control surface (actuator and engine) faults, with a total of 66 minutes of flight in normal conditions and 13 minutes of post-fault flight time. It additionally includes many hours of raw data of fully-autonomous, autopilot-assisted and manual flights with tens of fault scenarios. The ground truth of the time and type of faults is provided in each scenario to enable evaluation of the methods using the dataset. We have also provided the helper tools in several programming languages to load and work with the data and to help the evaluation of a detection method using the dataset. A set of metrics is proposed to help to compare different methods using the dataset. Most of the current fault detection methods are evaluated in simulation and as far as we know, this dataset is the only one providing the real flight data with faults in such capacity. We hope it will help advance the state-of-the-art in Anomaly Detection or FDI research for Autonomous Aerial Vehicles and mobile robots to enhance the safety of autonomous and remote flight operations further. The dataset and the provided tools can be accessed from http://theairlab.org/alfa-dataset.
JarKA: Modeling Attribute Interactions for Cross-lingual Knowledge Alignment
Authors: Bo Chen, Jing Zhang, Xiaobin Tang, Hong Chen, Cuiping Li
Date: 2019-10-29 00:00:00
Tasks: Attribute

Abstract. Cross-lingual knowledge alignment is the cornerstone in building a comprehensive knowledge graph (KG), which can benefit various knowledge-driven applications. As the structures of KGs are usually sparse, attributes of entities may play an important role in aligning the entities. However, the heterogeneity of the attributes across KGs prevents from accurately embedding and comparing entities. To deal with the issue, we propose to model the interactions between attributes, instead of globally embedding an entity with all the attributes. We further propose a joint framework to merge the alignments inferred from the attributes and the structures. Experimental results show that the proposed model outperforms the state-of-art baselines by up to 38.48% HitRatio@1. The results also demonstrate that our model can infer the alignments between attributes, relationships and values, in addition to entities.
Deep convolutional neural network application on rooftop detection for aerial image
Authors: Mengge Chen, Jonathan Li
Date: 2019-10-29 00:00:00

As one of the most destructive disasters in the world, earthquake causes death, injuries, destruction and enormous damage to the affected area. It is significant to detect buildings after an earthquake in response to reconstruction and damage evaluation. In this research, we proposed an automatic rooftop detection method based on the convolutional neural network (CNN) to extract buildings in the city of Christchurch and tuned hyperparameters to detect small detached houses from the aerial image. The experiment result shows that our approach can effectively and accurately detect and segment buildings and has competitive performance.
A Hierarchical Mixture Density Network
Authors: Fan Yang, Jaymar Soriano, Takatomi Kubo, Kazushi Ikeda
Date: 2019-10-23 00:00:00

The relationship among three correlated variables could be very sophisticated, as a result, we may not be able to find their hidden causality and model their relationship explicitly. However, we still can make our best guess for possible mappings among these variables, based on the observed relationship. One of the complicated relationships among three correlated variables could be a two-layer hierarchical many-to-many mapping. In this paper, we proposed a Hierarchical Mixture Density Network (HMDN) to model the two-layer hierarchical many-to-many mapping. We apply HMDN on an indoor positioning problem and show its benefit.
Domain Generalization via Model-Agnostic Learning of Semantic Features
Authors: Qi Dou, Daniel C. Castro, Konstantinos Kamnitsas, Ben Glocker
Date: 2019-10-29 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Domain Generalization, General Knowledge, Image Segmentation, Medical Image Segmentation, Metric Learning, Object Recognition, Semantic Segmentation

Generalization capability to unseen domains is crucial for machine learning models when deploying to real-world conditions. We investigate the challenging problem of domain generalization, i.e., training a model on multi-domain source data such that it can directly generalize to target domains with unknown statistics. We adopt a model-agnostic learning paradigm with gradient-based meta-train and meta-test procedures to expose the optimization to domain shift. Further, we introduce two complementary losses which explicitly regularize the semantic structure of the feature space. Globally, we align a derived soft confusion matrix to preserve general knowledge about inter-class relationships. Locally, we promote domain-independent class-specific cohesion and separation of sample features with a metric-learning component. The effectiveness of our method is demonstrated with new state-of-the-art results on two common object recognition benchmarks. Our method also shows consistent improvement on a medical image segmentation task.
Facial Image Deformation Based on Landmark Detection
Authors: Chaoyue Song, Yugang Chen, Shulai Zhang, Bingbing Ni
Date: 2019-10-30 00:00:00

In this work, we use facial landmarks to make the deformation for facial images more authentic. The deformation includes the expansion of eyes and the shrinking of noses, mouths, and cheeks. An advanced 106-point facial landmark detector is utilized to provide control points for deformation. Bilinear interpolation is used in the expansion and Moving Least Squares methods (MLS) including Affine Deformation, Similarity Deformation and Rigid Deformation are used in the shrinking. We compare the running time as well as the quality of deformed images using different MLS methods. The experimental results show that the Rigid Deformation which can keep other parts of the images unchanged performs better even if it takes the longest time.
Multi Modal Semantic Segmentation using Synthetic Data
Authors: Kartik Srivastava, Akash Kumar Singh, Guruprasad M. Hegde
Date: 2019-10-30 00:00:00
Tasks: Autonomous Driving, Semantic Segmentation

Semantic understanding of scenes in three-dimensional space (3D) is a quintessential part of robotics oriented applications such as autonomous driving as it provides geometric cues such as size, orientation and true distance of separation to objects which are crucial for taking mission critical decisions. As a first step, in this work we investigate the possibility of semantically classifying different parts of a given scene in 3D by learning the underlying geometric context in addition to the texture cues BUT in the absence of labelled real-world datasets. To this end we generate a large number of synthetic scenes, their pixel-wise labels and corresponding 3D representations using CARLA software framework. We then build a deep neural network that learns underlying category specific 3D representation and texture cues from color information of the rendered synthetic scenes. Further on we apply the learned model on different real world datasets to evaluate its performance. Our preliminary investigation of results show that the neural network is able to learn the geometric context from synthetic scenes and effectively apply this knowledge to classify each point of a 3D representation of a scene in real-world.
Dual Illumination Estimation for Robust Exposure Correction
Authors: Qing Zhang, Yongwei Nie, Wei-Shi Zheng
Date: 2019-10-30 00:00:00
Tasks: Exposure Correction, Multi-Exposure Image Fusion

Exposure correction is one of the fundamental tasks in image processing and computational photography. While various methods have been proposed, they either fail to produce visually pleasing results, or only work well for limited types of image (e.g., underexposed images). In this paper, we present a novel automatic exposure correction method, which is able to robustly produce high-quality results for images of various exposure conditions (e.g., underexposed, overexposed, and partially under- and over-exposed). At the core of our approach is the proposed dual illumination estimation, where we separately cast the under- and over-exposure correction as trivial illumination estimation of the input image and the inverted input image. By performing dual illumination estimation, we obtain two intermediate exposure correction results for the input image, with one fixes the underexposed regions and the other one restores the overexposed regions. A multi-exposure image fusion technique is then employed to adaptively blend the visually best exposed parts in the two intermediate exposure correction images and the input image into a globally well-exposed image. Experiments on a number of challenging images demonstrate the effectiveness of the proposed approach and its superiority over the state-of-the-art methods and popular automatic exposure correction tools.