pwc-archive/papers-with-abstracts huggingface dataset:
Deep Convolutions for In-Depth Automated Rock Typing
Authors: E. E. Baraboshkin, L. S. Ismailova, D. M. Orlov, E. A. Zhukovskaya, G. A. Kalmykov, O. V. Khotylev, E. Yu. Baraboshkin, D. A. Koroteev
Date: 2019-09-23 00:00:00

The description of rocks is one of the most time-consuming tasks in the everyday work of a geologist, especially when very accurate description is required. We here present a method that reduces the time needed for accurate description of rocks, enabling the geologist to work more efficiently. We describe the application of methods based on color distribution analysis and feature extraction. Then we focus on a new approach, used by us, which is based on convolutional neural networks. We used several well-known neural network architectures (AlexNet, VGG, GoogLeNet, ResNet) and made a comparison of their performance. The precision of the algorithms is up to 95% on the validation set with GoogLeNet architecture. The best of the proposed algorithms can describe 50 m of full-size core in one minute.
Scheduled Differentiable Architecture Search for Visual Recognition
Authors: Zhaofan Qiu, Ting Yao, Yiheng Zhang, Yongdong Zhang, Tao Mei
Date: 2019-09-23 00:00:00
Tasks: Video Recognition

Convolutional Neural Networks (CNN) have been regarded as a capable class of models for visual recognition problems. Nevertheless, it is not trivial to develop generic and powerful network architectures, which requires significant efforts of human experts. In this paper, we introduce a new idea for automatically exploring architectures on a remould of Differentiable Architecture Search (DAS), which possesses the efficient search via gradient descent. Specifically, we present Scheduled Differentiable Architecture Search (SDAS) for both image and video recognition that nicely integrates the selection of operations during training with a schedule. Technically, an architecture or a cell is represented as a directed graph. Our SDAS gradually fixes the operations on the edges in the graph in a progressive and scheduled manner, as opposed to a one-step decision of operations for all the edges once the training completes in existing DAS, which may make the architecture brittle. Moreover, we enlarge the search space of SDAS particularly for video recognition by devising several unique operations to encode spatio-temporal dynamics and demonstrate the impact in affecting the architecture search of SDAS. Extensive experiments of architecture learning are conducted on CIFAR10, Kinetics10, UCF101 and HMDB51 datasets, and superior results are reported when comparing to DAS method. More remarkably, the search by our SDAS is around 2-fold faster than DAS. When transferring the learnt cells on CIFAR10 and Kinetics10 respectively to large-scale ImageNet and Kinetics400 datasets, the constructed network also outperforms several state-of-the-art hand-crafted structures.
Large Scale Joint Semantic Re-Localisation and Scene Understanding via Globally Unique Instance Coordinate Regression
Authors: Ignas Budvytis, Marvin Teichmann, Tomas Vojir, Roberto Cipolla
Date: 2019-09-23 00:00:00
Tasks: 3D geometry, Autonomous Driving, Camera Pose Estimation, Pose Estimation, regression, Scene Understanding

In this work we present a novel approach to joint semantic localisation and scene understanding. Our work is motivated by the need for localisation algorithms which not only predict 6-DoF camera pose but also simultaneously recognise surrounding objects and estimate 3D geometry. Such capabilities are crucial for computer vision guided systems which interact with the environment: autonomous driving, augmented reality and robotics. In particular, we propose a two step procedure. During the first step we train a convolutional neural network to jointly predict per-pixel globally unique instance labels and corresponding local coordinates for each instance of a static object (e.g. a building). During the second step we obtain scene coordinates by combining object center coordinates and local coordinates and use them to perform 6-DoF camera pose estimation. We evaluate our approach on real world (CamVid-360) and artificial (SceneCity) autonomous driving datasets. We obtain smaller mean distance and angular errors than state-of-the-art 6-DoF pose estimation algorithms based on direct pose regression and pose estimation from scene coordinates on all datasets. Our contributions include: (i) a novel formulation of scene coordinate regression as two separate tasks of object instance recognition and local coordinate regression and a demonstration that our proposed solution allows to predict accurate 3D geometry of static objects and estimate 6-DoF pose of camera on (ii) maps larger by several orders of magnitude than previously attempted by scene coordinate regression methods, as well as on (iii) lightweight, approximate 3D maps built from 3D primitives such as building-aligned cuboids.
Predicting Landscapes from Environmental Conditions Using Generative Networks
Authors: Christian Requena-Mesa, Markus Reichstein, Miguel Mahecha, Basil Kraft, Joachim Denzler
Date: 2019-09-23 00:00:00
Tasks: Generative Adversarial Network

Landscapes are meaningful ecological units that strongly depend on the environmental conditions. Such dependencies between landscapes and the environment have been noted since the beginning of Earth sciences and cast into conceptual models describing the interdependencies of climate, geology, vegetation and geomorphology. Here, we ask whether landscapes, as seen from space, can be statistically predicted from pertinent environmental conditions. To this end we adapted a deep learning generative model in order to establish the relationship between the environmental conditions and the view of landscapes from the Sentinel-2 satellite. We trained a conditional generative adversarial network to generate multispectral imagery given a set of climatic, terrain and anthropogenic predictors. The generated imagery of the landscapes share many characteristics with the real one. Results based on landscape patch metrics, indicative of landscape composition and structure, show that the proposed generative model creates landscapes that are more similar to the targets than the baseline models while overall reflectance and vegetation cover are predicted better. We demonstrate that for many purposes the generated landscapes behave as real with immediate application for global change studies. We envision the application of machine learning as a tool to forecast the effects of climate change on the spatial features of landscapes, while we assess its limitations and breaking points.
Where to Look Next: Unsupervised Active Visual Exploration on 360° Input
Authors: Soroush Seifi, Tinne Tuytelaars
Date: 2019-09-23 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning, Reinforcement Learning (RL)

We address the problem of active visual exploration of large 360{\deg} inputs. In our setting an active agent with a limited camera bandwidth explores its 360{\deg} environment by changing its viewing direction at limited discrete time steps. As such, it observes the world as a sequence of narrow field-of-view 'glimpses', deciding for itself where to look next. Our proposed method exceeds previous works' performance by a significant margin without the need for deep reinforcement learning or training separate networks as sidekicks. A key component of our system are the spatial memory maps that make the system aware of the glimpses' orientations (locations in the 360{\deg} image). Further, we stress the advantages of retina-like glimpses when the agent's sensor bandwidth and time-steps are limited. Finally, we use our trained model to do classification of the whole scene using only the information observed in the glimpses.
Human Synthesis and Scene Compositing
Authors: Mihai Zanfir, Elisabeta Oneata, Alin-Ionut Popa, Andrei Zanfir, Cristian Sminchisescu
Date: 2019-09-23 00:00:00
Tasks: Image Compression, Image Generation, Virtual Try-on

Generating good quality and geometrically plausible synthetic images of humans with the ability to control appearance, pose and shape parameters, has become increasingly important for a variety of tasks ranging from photo editing, fashion virtual try-on, to special effects and image compression. In this paper, we propose HUSC, a HUman Synthesis and Scene Compositing framework for the realistic synthesis of humans with different appearance, in novel poses and scenes. Central to our formulation is 3d reasoning for both people and scenes, in order to produce realistic collages, by correctly modeling perspective effects and occlusion, by taking into account scene semantics and by adequately handling relative scales. Conceptually our framework consists of three components: (1) a human image synthesis model with controllable pose and appearance, based on a parametric representation, (2) a person insertion procedure that leverages the geometry and semantics of the 3d scene, and (3) an appearance compositing process to create a seamless blending between the colors of the scene and the generated human image, and avoid visual artifacts. The performance of our framework is supported by both qualitative and quantitative results, in particular state-of-the art synthesis scores for the DeepFashion dataset.
How to improve CNN-based 6-DoF camera pose estimation
Authors: Soroush Seifi, Tinne Tuytelaars
Date: 2019-09-23 00:00:00
Tasks: Camera Pose Estimation, Data Augmentation, Pose Estimation, Transfer Learning

Convolutional neural networks (CNNs) and transfer learning have recently been used for 6 degrees of freedom (6-DoF) camera pose estimation. While they do not reach the same accuracy as visual SLAM-based approaches and are restricted to a specific environment, they excel in robustness and can be applied even to a single image. In this paper, we study PoseNet [1] and investigate modifications based on datasets' characteristics to improve the accuracy of the pose estimates. In particular, we emphasize the importance of field-of-view over image resolution; we present a data augmentation scheme to reduce overfitting; we study the effect of Long-Short-Term-Memory (LSTM) cells. Lastly, we combine these modifications and improve PoseNet's performance for monocular CNN based camera pose regression.
Object Segmentation using Pixel-wise Adversarial Loss
Authors: Ricard Durall, Franz-Josef Pfreundt, Ullrich Köthe, Janis Keuper
Date: 2019-09-23 00:00:00
Tasks: Object, Segmentation, Semantic Segmentation

Recent deep learning based approaches have shown remarkable success on object segmentation tasks. However, there is still room for further improvement. Inspired by generative adversarial networks, we present a generic end-to-end adversarial approach, which can be combined with a wide range of existing semantic segmentation networks to improve their segmentation performance. The key element of our method is to replace the commonly used binary adversarial loss with a high resolution pixel-wise loss. In addition, we train our generator employing stochastic weight averaging fashion, which further enhances the predicted output label maps leading to state-of-the-art results. We show, that this combination of pixel-wise adversarial training and weight averaging leads to significant and consistent gains in segmentation performance, compared to the baseline models.
RAUNet: Residual Attention U-Net for Semantic Segmentation of Cataract Surgical Instruments
Authors: Zhen-Liang Ni, Gui-Bin Bian, Xiao-Hu Zhou, Zeng-Guang Hou, Xiao-Liang Xie, Chen Wang, Yan-Jie Zhou, Rui-Qi Li, Zhen Li
Date: 2019-09-23 00:00:00
Tasks: Segmentation, Semantic Segmentation

Semantic segmentation of surgical instruments plays a crucial role in robot-assisted surgery. However, accurate segmentation of cataract surgical instruments is still a challenge due to specular reflection and class imbalance issues. In this paper, an attention-guided network is proposed to segment the cataract surgical instrument. A new attention module is designed to learn discriminative features and address the specular reflection issue. It captures global context and encodes semantic dependencies to emphasize key semantic features, boosting the feature representation. This attention module has very few parameters, which helps to save memory. Thus, it can be flexibly plugged into other networks. Besides, a hybrid loss is introduced to train our network for addressing the class imbalance issue, which merges cross entropy and logarithms of Dice loss. A new dataset named Cata7 is constructed to evaluate our network. To the best of our knowledge, this is the first cataract surgical instrument dataset for semantic segmentation. Based on this dataset, RAUNet achieves state-of-the-art performance 97.71% mean Dice and 95.62% mean IOU.
Shadow Transfer: Single Image Relighting For Urban Road Scenes
Authors: Alexandra Carlson, Ram Vasudevan, Matthew Johnson-Roberson
Date: 2019-09-23 00:00:00
Tasks: Diversity, Image Relighting, Image-to-Image Translation, Translation

Illumination effects in images, specifically cast shadows and shading, have been shown to decrease the performance of deep neural networks on a large number of vision-based detection, recognition and segmentation tasks in urban driving scenes. A key factor that contributes to this performance gap is the lack of `time-of-day' diversity within real, labeled datasets. There have been impressive advances in the realm of image to image translation in transferring previously unseen visual effects into a dataset, specifically in day to night translation. However, it is not easy to constrain what visual effects, let alone illumination effects, are transferred from one dataset to another during the training process. To address this problem, we propose deep learning framework, called Shadow Transfer, that can relight complex outdoor scenes by transferring realistic shadow, shading, and other lighting effects onto a single image. The novelty of the proposed framework is that it is both self-supervised, and is designed to operate on sensor and label information that is easily available in autonomous vehicle datasets. We show the effectiveness of this method on both synthetic and real datasets, and we provide experiments that demonstrate that the proposed method produces images of higher visual quality than state of the art image to image translation methods.
Model-Based and Data-Driven Strategies in Medical Image Computing
Authors: Daniel Rueckert, Julia A. Schnabel
Date: 2019-09-23 00:00:00
Tasks: Image Reconstruction, Transfer Learning

Model-based approaches for image reconstruction, analysis and interpretation have made significant progress over the last decades. Many of these approaches are based on either mathematical, physical or biological models. A challenge for these approaches is the modelling of the underlying processes (e.g. the physics of image acquisition or the patho-physiology of a disease) with appropriate levels of detail and realism. With the availability of large amounts of imaging data and machine learning (in particular deep learning) techniques, data-driven approaches have become more widespread for use in different tasks in reconstruction, analysis and interpretation. These approaches learn statistical models directly from labelled or unlabeled image data and have been shown to be very powerful for extracting clinically useful information from medical imaging. While these data-driven approaches often outperform traditional model-based approaches, their clinical deployment often poses challenges in terms of robustness, generalization ability and interpretability. In this article, we discuss what developments have motivated the shift from model-based approaches towards data-driven strategies and what potential problems are associated with the move towards purely data-driven approaches, in particular deep learning. We also discuss some of the open challenges for data-driven approaches, e.g. generalization to new unseen data (e.g. transfer learning), robustness to adversarial attacks and interpretability. Finally, we conclude with a discussion on how these approaches may lead to the development of more closely coupled imaging pipelines that are optimized in an end-to-end fashion.
Go Wider: An Efficient Neural Network for Point Cloud Analysis via Group Convolutions
Authors: Can Chen, Luca Zanotti Fragonara, Antonios Tsourdos
Date: 2019-09-23 00:00:00
Tasks: Autonomous Driving, Efficient Neural Network, Semantic Segmentation

In order to achieve better performance for point cloud analysis, many researchers apply deeper neural networks using stacked Multi-Layer-Perceptron (MLP) convolutions over irregular point cloud. However, applying dense MLP convolutions over large amount of points (e.g. autonomous driving application) leads to inefficiency in memory and computation. To achieve high performance but less complexity, we propose a deep-wide neural network, called ShufflePointNet, to exploit fine-grained local features and reduce redundancy in parallel using group convolution and channel shuffle operation. Unlike conventional operation that directly applies MLPs on high-dimensional features of point cloud, our model goes wider by splitting features into groups in advance, and each group with certain smaller depth is only responsible for respective MLP operation, which can reduce complexity and allows to encode more useful information. Meanwhile, we connect communication between groups by shuffling groups in feature channel to capture fine-grained features. We claim that, multi-branch method for wider neural networks is also beneficial to feature extraction for point cloud. We present extensive experiments for shape classification task on ModelNet40 dataset and semantic segmentation task on large scale datasets ShapeNet part, S3DIS and KITTI. We further perform ablation study and compare our model to other state-of-the-art algorithms in terms of complexity and accuracy.
Patch-Based Image Similarity for Intraoperative 2D/3D Pelvis Registration During Periacetabular Osteotomy
Authors: Robert Grupp, Mehran Armand, Russell Taylor
Date: 2019-09-23 00:00:00

Periacetabular osteotomy is a challenging surgical procedure for treating developmental hip dysplasia, providing greater coverage of the femoral head via relocation of a patient's acetabulum. Since fluoroscopic imaging is frequently used in the surgical workflow, computer-assisted X-Ray navigation of osteotomes and the relocated acetabular fragment should be feasible. We use intensity-based 2D/3D registration to estimate the pelvis pose with respect to fluoroscopic images, recover relative poses of multiple views, and triangulate landmarks which may be used for navigation. Existing similarity metrics are unable to consistently account for the inherent mismatch between the preoperative intact pelvis, and the intraoperative reality of a fractured pelvis. To mitigate the effect of this mismatch, we continuously estimate the relevance of each pixel to solving the registration and use these values as weightings in a patch-based similarity metric. Limiting computation to randomly selected subsets of patches results in faster runtimes than existing patch-based methods. A simulation study was conducted with random fragment shapes, relocations, and fluoroscopic views, and the proposed method achieved a 1.7 mm mean triangulation error over all landmarks, compared to mean errors of 3 mm and 2.8 mm for the non-patched and image-intensity-variance-weighted patch similarity metrics, respectively.
Pelvis Surface Estimation From Partial CT for Computer-Aided Pelvic Osteotomies
Authors: Robert Grupp, Yoshito Otake, Ryan Murphy, Javad Parvizi, Mehran Armand, Russell Taylor
Date: 2019-09-23 00:00:00
Tasks: Anatomy

Computer-aided surgical systems commonly use preoperative CT scans when performing pelvic osteotomies for intraoperative navigation. These systems have the potential to improve the safety and accuracy of pelvic osteotomies, however, exposing the patient to radiation is a significant drawback. In order to reduce radiation exposure, we propose a new smooth extrapolation method leveraging a partial pelvis CT and a statistical shape model (SSM) of the full pelvis in order to estimate a patient's complete pelvis. A SSM of normal, complete, female pelvis anatomy was created and evaluated from 42 subjects. A leave-one-out test was performed to characterise the inherent generalisation capability of the SSM. An additional leave-one-out test was conducted to measure performance of the smooth extrapolation method and an existing "cut-and-paste" extrapolation method. Unknown anatomy was simulated by keeping the axial slices of the patient's acetabulum intact and varying the amount of the superior iliac crest retained; from 0% to 15% of the total pelvis extent. The smooth technique showed an average improvement over the cut-and-paste method of 1.31 mm and 3.61 mm, in RMS and maximum surface error, respectively. With 5% of the iliac crest retained, the smoothly estimated surface had an RMS surface error of 2.21 mm, an improvement of 1.25 mm when retaining none of the iliac crest. This anatomical estimation method creates the possibility of a patient and surgeon benefiting from the use of a CAS system and simultaneously reducing the patient's radiation exposure.
Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation
Authors: Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, Jiaya Jia
Date: 2019-09-23 00:00:00
Proceeding: ICCV 2019 10
Tasks: Decoder, Scene Labeling, Semantic Segmentation

We achieve 3D semantic scene labeling by exploring semantic relation between each point and its contextual neighbors through edges. Besides an encoder-decoder branch for predicting point labels, we construct an edge branch to hierarchically integrate point features and generate edge features. To incorporate point features in the edge branch, we establish a hierarchical graph framework, where the graph is initialized from a coarse layer and gradually enriched along the point decoding process. For each edge in the final graph, we predict a label to indicate the semantic consistency of the two connected points to enhance point prediction. At different layers, edge features are also fed into the corresponding point module to integrate contextual information for message passing enhancement in local regions. The two branches interact with each other and cooperate in segmentation. Decent experimental results on several 3D semantic labeling datasets demonstrate the effectiveness of our work.
Hydrocephalus verification on brain magnetic resonance images with deep convolutional neural networks and "transfer learning" technique
Authors: Alexey Demyanchuk, Ekaterina Pushkina, Nikolay Russkikh, Dmitry Shtokalo, Sergey Mishinov
Date: 2019-09-23 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Specificity, Transfer Learning

The hydrocephalus can be either an independent disease or a concomitant symptom of a number of pathologies, therefore representing an urgent issue in the present-day clinical practice. Deep Learning is an evolving technology and the part of a broader field of Machine Learning. Deep learning is currently actively researched in the field of radiology. The aim of this study was to evaluate deep learning applicability to the diagnostics of hydrocephalus with the use of MRI images. We retrospectively collected, annotated, and preprocessed the brain MRI data of 200 patients with and without radiological signs of hydrocephalus. We applied a state-of-the-art deep convolutional neural network in conjunction with transfer learning method to train a hydrocephalus classifier model. Using deep convolutional neural networks, we achieved a high quality of machine learning model. Accuracy, sensitivity, and specificity of hydrocephalus signs identification was 97%, 98%, and 96% respectively. In this study, we demonstrated the capacity of deep neural networks to identify hydrocephalus syndrome using brain MRI images. Applying transfer learning technique, the high quality of classification was achieved although trained on rather limited data.