pwc-archive/papers-with-abstracts huggingface dataset:
Multi-modality Latent Interaction Network for Visual Question Answering
Authors: Peng Gao, Haoxuan You, Zhanpeng Zhang, Xiaogang Wang, Hongsheng Li
Date: 2019-08-10 00:00:00
Proceeding: ICCV 2019 10
Tasks: Language Modeling, Language Modelling, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Exploiting relationships between visual regions and question words have achieved great success in learning multi-modality features for Visual Question Answering (VQA). However, we argue that existing methods mostly model relations between individual visual regions and words, which are not enough to correctly answer the question. From humans' perspective, answering a visual question requires understanding the summarizations of visual and language information. In this paper, we proposed the Multi-modality Latent Interaction module (MLI) to tackle this problem. The proposed module learns the cross-modality relationships between latent visual and language summarizations, which summarize visual regions and question into a small number of latent representations to avoid modeling uninformative individual region-word relations. The cross-modality information between the latent summarizations are propagated to fuse valuable information from both modalities and are used to update the visual and word features. Such MLI modules can be stacked for several stages to model complex and latent relations between the two modalities and achieves highly competitive performance on public VQA benchmarks, VQA v2.0 and TDIUC . In addition, we show that the performance of our methods could be significantly improved by combining with pre-trained language model BERT.
Identification of relevant diffusion MRI metrics impacting cognitive functions using a novel feature selection method
Authors: Tongda Xu, Xiyan Cai, Yao Wang, Xiuyuan Wang, Sohae Chung, Els Fieremans, Joseph Rath, Steven Flanagan, Yvonne W. Lui
Date: 2019-08-10 00:00:00
Tasks: Diffusion  MRI, feature selection

Mild Traumatic Brain Injury (mTBI) is a significant public health problem. The most troubling symptoms after mTBI are cognitive complaints. Studies show measurable differences between patients with mTBI and healthy controls with respect to tissue microstructure using diffusion MRI. However, it remains unclear which diffusion measures are the most informative with regard to cognitive functions in both the healthy state as well as after injury. In this study, we use diffusion MRI to formulate a predictive model for performance on working memory based on the most relevant MRI features. The key challenge is to identify relevant features over a large feature space with high accuracy in an efficient manner. To tackle this challenge, we propose a novel improvement of the best first search approach with crossover operators inspired by genetic algorithm. Compared against other heuristic feature selection algorithms, the proposed method achieves significantly more accurate predictions and yields clinically interpretable selected features.
The Channel Attention based Context Encoder Network for Inner Limiting Membrane Detection
Authors: Hao Qiu, Zaiwang Gu, Lei Mou, Xiaoqian Mao, Liyang Fang, Yitian Zhao, Jiang Liu, Jun Cheng
Date: 2019-08-09 00:00:00
Tasks: Decoder, Optic Disc Segmentation, Segmentation

The optic disc segmentation is an important step for retinal image-based disease diagnosis such as glaucoma. The inner limiting membrane (ILM) is the first boundary in the OCT, which can help to extract the retinal pigment epithelium (RPE) through gradient edge information to locate the boundary of the optic disc. Thus, the ILM layer segmentation is of great importance for optic disc localization. In this paper, we build a new optic disc centered dataset from 20 volunteers and manually annotated the ILM boundary in each OCT scan as ground-truth. We also propose a channel attention based context encoder network modified from the CE-Net to segment the optic disc. It mainly contains three phases: the encoder module, the channel attention based context encoder module, and the decoder module. Finally, we demonstrate that our proposed method achieves state-of-the-art disc segmentation performance on our dataset mentioned above.
Repetitive Reprediction Deep Decipher for Semi-Supervised Learning
Authors: Guo-Hua Wang, Jianxin Wu
Date: 2019-08-09 00:00:00
Tasks: Semi-Supervised Image Classification

Most recent semi-supervised deep learning (deep SSL) methods used a similar paradigm: use network predictions to update pseudo-labels and use pseudo-labels to update network parameters iteratively. However, they lack theoretical support and cannot explain why predictions are good candidates for pseudo-labels. In this paper, we propose a principled end-to-end framework named deep decipher (D2) for SSL. Within the D2 framework, we prove that pseudo-labels are related to network predictions by an exponential link function, which gives a theoretical support for using predictions as pseudo-labels. Furthermore, we demonstrate that updating pseudo-labels by network predictions will make them uncertain. To mitigate this problem, we propose a training strategy called repetitive reprediction (R2). Finally, the proposed R2-D2 method is tested on the large-scale ImageNet dataset and outperforms state-of-the-art methods by 5 percentage points.
Enforcing Perceptual Consistency on Generative Adversarial Networks by Using the Normalised Laplacian Pyramid Distance
Authors: Alexander Hepburn, Valero Laparra, Ryan McConville, Raul Santos-Rodriguez
Date: 2019-08-09 00:00:00
Tasks: Image Generation, Image Segmentation, Image-to-Image Translation, Semantic Segmentation, Translation

In recent years there has been a growing interest in image generation through deep learning. While an important part of the evaluation of the generated images usually involves visual inspection, the inclusion of human perception as a factor in the training process is often overlooked. In this paper we propose an alternative perceptual regulariser for image-to-image translation using conditional generative adversarial networks (cGANs). To do so automatically (avoiding visual inspection), we use the Normalised Laplacian Pyramid Distance (NLPD) to measure the perceptual similarity between the generated image and the original image. The NLPD is based on the principle of normalising the value of coefficients with respect to a local estimate of mean energy at different scales and has already been successfully tested in different experiments involving human perception. We compare this regulariser with the originally proposed L1 distance and note that when using NLPD the generated images contain more realistic values for both local and global contrast. We found that using NLPD as a regulariser improves image segmentation accuracy on generated images as well as improving two no-reference image quality metrics.
Graph Embedding Using Infomax for ASD Classification and Brain Functional Difference Detection
Authors: Xiaoxiao Li, Nicha C. Dvornek, Juntang Zhuang, Pamela Ventola, James Duncan
Date: 2019-08-09 00:00:00
Tasks: Classification, General Classification, Graph Embedding, Graph Neural Network

Significant progress has been made using fMRI to characterize the brain changes that occur in ASD, a complex neuro-developmental disorder. However, due to the high dimensionality and low signal-to-noise ratio of fMRI, embedding informative and robust brain regional fMRI representations for both graph-level classification and region-level functional difference detection tasks between ASD and healthy control (HC) groups is difficult. Here, we model the whole brain fMRI as a graph, which preserves geometrical and temporal information and use a Graph Neural Network (GNN) to learn from the graph-structured fMRI data. We investigate the potential of including mutual information (MI) loss (Infomax), which is an unsupervised term encouraging large MI of each nodal representation and its corresponding graph-level summarized representation to learn a better graph embedding. Specifically, this work developed a pipeline including a GNN encoder, a classifier and a discriminator, which forces the encoded nodal representations to both benefit classification and reveal the common nodal patterns in a graph. We simultaneously optimize graph-level classification loss and Infomax. We demonstrated that Infomax graph embedding improves classification performance as a regularization term. Furthermore, we found separable nodal representations of ASD and HC groups in prefrontal cortex, cingulate cortex, visual regions, and other social, emotional and execution related brain regions. In contrast with GNN with classification loss only, the proposed pipeline can facilitate training more robust ASD classification models. Moreover, the separable nodal representations can detect the functional differences between the two groups and contribute to revealing new ASD biomarkers.
Apache Spark Accelerated Deep Learning Inference for Large Scale Satellite Image Analytics
Authors: Dalton Lunga, Jonathan Gerrand, Hsiuhan Lexie Yang, Christopher Layton, Robert Stewart
Date: 2019-08-08 00:00:00
Tasks: Deep Learning, Earth Observation, GPU, Semantic Segmentation

The shear volumes of data generated from earth observation and remote sensing technologies continue to make major impact; leaping key geospatial applications into the dual data and compute intensive era. As a consequence, this rapid advancement poses new computational and data processing challenges. We implement a novel remote sensing data flow (RESFlow) for advanced machine learning and computing with massive amounts of remotely sensed imagery. The core contribution is partitioning massive amount of data based on the spectral and semantic characteristics for distributed imagery analysis. RESFlow takes advantage of both a unified analytics engine for large-scale data processing and the availability of modern computing hardware to harness the acceleration of deep learning inference on expansive remote sensing imagery. The framework incorporates a strategy to optimize resource utilization across multiple executors assigned to a single worker. We showcase its deployment across computationally and data-intensive on pixel-level labeling workloads. The pipeline invokes deep learning inference at three stages; during deep feature extraction, deep metric mapping, and deep semantic segmentation. The tasks impose compute intensive and GPU resource sharing challenges motivating for a parallelized pipeline for all execution steps. By taking advantage of Apache Spark, Nvidia DGX1, and DGX2 computing platforms, we demonstrate unprecedented compute speed-ups for deep learning inference on pixel labeling workloads; processing 21,028~Terrabytes of imagery data and delivering an output maps at area rate of 5.245sq.km/sec, amounting to 453,168 sq.km/day - reducing a 28 day workload to 21~hours.
Fine-Tuning Models Comparisons on Garbage Classification for Recyclability
Authors: Umut Ozkaya, Levent Seyfi
Date: 2019-08-07 00:00:00
Tasks: Classification, General Classification, Transfer Learning

In this study, it is aimed to develop a deep learning application which detects types of garbage into trash in order to provide recyclability with vision system. Training and testing will be performed with image data consisting of several classes on different garbage types. The data set used during training and testing will be generated from original frames taken from garbage images. The data set used for deep learning structures has a total of 2527 images with 6 different classes. Half of these images in the data set were used for training process and remaining part were used for testing procedure. Also, transfer learning was used to obtain shorter training and test procedures with and higher accuracy. As fine-tuned models, Alexnet, VGG16, Googlenet and Resnet structures were carried. In order to test performance of classifiers, two different classifiers are used as Softmax and Support Vector Machines. 6 different type of trash images were correctly classified the highest accuracy with GoogleNet+SVM as 97.86%.
A fast multi-object tracking system using an object detector ensemble
Authors: Richard Cobos, Jefferson Hernandez, Andres G. Abad
Date: 2019-08-06 00:00:00
Tasks: Multi-Object Tracking, Multiple Object Tracking, Object, Object Tracking

Multiple-Object Tracking (MOT) is of crucial importance for applications such as retail video analytics and video surveillance. Object detectors are often the computational bottleneck of modern MOT systems, limiting their use for real-time applications. In this paper, we address this issue by leveraging on an ensemble of detectors, each running every f frames. We measured the performance of our system in the MOT16 benchmark. The proposed model surpassed other online entries of the MOT16 challenge in speed, while maintaining an acceptable accuracy.
An Unsupervised, Iterative N-Dimensional Point-Set Registration Algorithm
Authors: A. Pasha Hosseinbor, R. Zhdanov, A. Ushveridze
Date: 2019-08-06 00:00:00

An unsupervised, iterative point-set registration algorithm for an unlabeled (i.e. correspondence between points is unknown) N-dimensional Euclidean point-cloud is proposed. It is based on linear least squares, and considers all possible point pairings and iteratively aligns the two sets until the number of point pairs does not exceed the maximum number of allowable one-to-one pairings.
Deep Learning for Detecting Building Defects Using Convolutional Neural Networks
Authors: Husein Perez, Joseph H. M. Tah, Amir Mosavi
Date: 2019-08-06 00:00:00
Tasks: Deep Learning

Clients are increasingly looking for fast and effective means to quickly and frequently survey and communicate the condition of their buildings so that essential repairs and maintenance work can be done in a proactive and timely manner before it becomes too dangerous and expensive. Traditional methods for this type of work commonly comprise of engaging building surveyors to undertake a condition assessment which involves a lengthy site inspection to produce a systematic recording of the physical condition of the building elements, including cost estimates of immediate and projected long-term costs of renewal, repair and maintenance of the building. Current asset condition assessment procedures are extensively time consuming, laborious, and expensive and pose health and safety threats to surveyors, particularly at height and roof levels which are difficult to access. This paper aims at evaluating the application of convolutional neural networks (CNN) towards an automated detection and localisation of key building defects, e.g., mould, deterioration, and stain, from images. The proposed model is based on pre-trained CNN classifier of VGG-16 (later compaired with ResNet-50, and Inception models), with class activation mapping (CAM) for object localisation. The challenges and limitations of the model in real-life applications have been identified. The proposed model has proven to be robust and able to accurately detect and localise building defects. The approach is being developed with the potential to scale-up and further advance to support automated detection of defects and deterioration of buildings in real-time using mobile devices and drones.
Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation
Authors: Brian Kenji Iwana, Ryohei Kuroki, Seiichi Uchida
Date: 2019-08-06 00:00:00
Tasks: Attribute, Classification, General Classification, image-classification, Image Classification, Multi-class Classification

Convolutional Neural Networks (CNN) have become state-of-the-art in the field of image classification. However, not everything is understood about their inner representations. This paper tackles the interpretability and explainability of the predictions of CNNs for multi-class classification problems. Specifically, we propose a novel visualization method of pixel-wise input attribution called Softmax-Gradient Layer-wise Relevance Propagation (SGLRP). The proposed model is a class discriminate extension to Deep Taylor Decomposition (DTD) using the gradient of softmax to back propagate the relevance of the output probability to the input image. Through qualitative and quantitative analysis, we demonstrate that SGLRP can successfully localize and attribute the regions on input images which contribute to a target object's classification. We show that the proposed method excels at discriminating the target objects class from the other possible objects in the images. We confirm that SGLRP performs better than existing Layer-wise Relevance Propagation (LRP) based methods and can help in the understanding of the decision process of CNNs.
SkrGAN: Sketching-rendering Unconditional Generative Adversarial Networks for Medical Image Synthesis
Authors: Tianyang Zhang, Huazhu Fu, Yitian Zhao, Jun Cheng, Mengjie Guo, Zaiwang Gu, Bing Yang, Yuting Xiao, Shenghua Gao, Jiang Liu
Date: 2019-08-06 00:00:00
Tasks: Computed Tomography (CT), Data Augmentation, Diagnostic, Generative Adversarial Network, Image Generation, Image Segmentation, Medical Image Analysis, Medical Image Generation, Medical Image Segmentation, Semantic Segmentation

Generative Adversarial Networks (GANs) have the capability of synthesizing images, which have been successfully applied to medical image synthesis tasks. However, most of existing methods merely consider the global contextual information and ignore the fine foreground structures, e.g., vessel, skeleton, which may contain diagnostic indicators for medical image analysis. Inspired by human painting procedure, which is composed of stroking and color rendering steps, we propose a Sketching-rendering Unconditional Generative Adversarial Network (SkrGAN) to introduce a sketch prior constraint to guide the medical image generation. In our SkrGAN, a sketch guidance module is utilized to generate a high quality structural sketch from random noise, then a color render mapping is used to embed the sketch-based representations and resemble the background appearances. Experimental results show that the proposed SkrGAN achieves the state-of-the-art results in synthesizing images for various image modalities, including retinal color fundus, X-Ray, Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). In addition, we also show that the performances of medical image segmentation method have been improved by using our synthesized images as data augmentation.
Local Supports Global: Deep Camera Relocalization with Sequence Enhancement
Authors: Fei Xue, Xin Wang, Zike Yan, Qiuyuan Wang, Junqiu Wang, Hongbin Zha
Date: 2019-08-06 00:00:00
Proceeding: ICCV 2019 10
Tasks: Camera Relocalization, Pose Estimation, Visual Odometry

We propose to leverage the local information in image sequences to support global camera relocalization. In contrast to previous methods that regress global poses from single images, we exploit the spatial-temporal consistency in sequential images to alleviate uncertainty due to visual ambiguities by incorporating a visual odometry (VO) component. Specifically, we introduce two effective steps called content-augmented pose estimation and motion-based refinement. The content-augmentation step focuses on alleviating the uncertainty of pose estimation by augmenting the observation based on the co-visibility in local maps built by the VO stream. Besides, the motion-based refinement is formulated as a pose graph, where the camera poses are further optimized by adopting relative poses provided by the VO component as additional motion constraints. Thus, the global consistency can be guaranteed. Experiments on the public indoor 7-Scenes and outdoor Oxford RobotCar benchmark datasets demonstrate that benefited from local information inherent in the sequence, our approach outperforms state-of-the-art methods, especially in some challenging cases, e.g., insufficient texture, highly repetitive textures, similar appearances, and over-exposure.
OD-GCN: Object Detection Boosted by Knowledge GCN
Authors: Zheng Liu, Zidong Jiang, Wei Feng, Hui Feng
Date: 2019-08-06 00:00:00
Tasks: Object, object-detection, Object Detection

Classical CNN based object detection methods only extract the objects' image features, but do not consider the high-level relationship among objects in context. In this article, the graph convolutional networks (GCN) is integrated into the object detection framework to exploit the benefit of category relationship among objects, which is able to provide extra confidence for any pre-trained object detection model in our framework. In experiments, we test several popular base detection models on COCO dataset. The results show promising improvement on mAP by 1-5pp. In addition, visualized analysis reveals the benchmark improvement is quite reasonable in human's opinion.
The Noise Collector for sparse recovery in high dimensions
Authors: Miguel Moscoso, Alexei Novikov, George Papanicolaou, Chrysoula Tsogka
Date: 2019-08-05 00:00:00
Tasks: parameter estimation, Vocal Bursts Intensity Prediction

The ability to detect sparse signals from noisy high-dimensional data is a top priority in modern science and engineering. A sparse solution of the linear system $A \rho = b_0$ can be found efficiently with an $l_1$-norm minimization approach if the data is noiseless. Detection of the signal's support from data corrupted by noise is still a challenging problem, especially if the level of noise must be estimated. We propose a new efficient approach that does not require any parameter estimation. We introduce the Noise Collector (NC) matrix $C$ and solve an augmented system $A \rho + C \eta = b_0 + e$, where $ e$ is the noise. We show that the $l_1$-norm minimal solution of the augmented system has zero false discovery rate for any level of noise and with probability that tends to one as the dimension of $ b_0$ increases to infinity. We also obtain exact support recovery if the noise is not too large, and develop a Fast Noise Collector Algorithm which makes the computational cost of solving the augmented system comparable to that of the original one. Finally, we demonstrate the effectiveness of the method in applications to passive array imaging.
Difficulty Classification of Mountainbike Downhill Trails utilizing Deep Neural Networks
Authors: Stefan Langer, Robert Müller, Kyrill Schmid, Claudia Linnhoff-Popien
Date: 2019-08-05 00:00:00
Tasks: Classification, General Classification

The difficulty of mountainbike downhill trails is a subjective perception. However, sports-associations and mountainbike park operators attempt to group trails into different levels of difficulty with scales like the Singletrail-Skala (S0-S5) or colored scales (blue, red, black, ...) as proposed by The International Mountain Bicycling Association. Inconsistencies in difficulty grading occur due to the various scales, different people grading the trails, differences in topography, and more. We propose an end-to-end deep learning approach to classify trails into three difficulties easy, medium, and hard by using sensor data. With mbientlab Meta Motion r0.2 sensor units, we record accelerometer- and gyroscope data of one rider on multiple trail segments. A 2D convolutional neural network is trained with a stacked and concatenated representation of the aforementioned data as its input. We run experiments with five different sample- and five different kernel sizes and achieve a maximum Sparse Categorical Accuracy of 0.9097. To the best of our knowledge, this is the first work targeting computational difficulty classification of mountainbike downhill trails.