pwc-archive/papers-with-abstracts huggingface dataset:
Predicting the Time Until a Vehicle Changes the Lane Using LSTM-based Recurrent Neural Networks
Authors: Florian Wirthmüller, Marvin Klimke, Julian Schlechtriemen, Jochen Hipp, Manfred Reichert
Date: 2021-02-02 00:00:00

To plan safe and comfortable trajectories for automated vehicles on highways, accurate predictions of traffic situations are needed. So far, a lot of research effort has been spent on detecting lane change maneuvers rather than on estimating the point in time a lane change actually happens. In practice, however, this temporal information might be even more useful. This paper deals with the development of a system that accurately predicts the time to the next lane change of surrounding vehicles on highways using long short-term memory-based recurrent neural networks. An extensive evaluation based on a large real-world data set shows that our approach is able to make reliable predictions, even in the most challenging situations, with a root mean squared error around 0.7 seconds. Already 3.5 seconds prior to lane changes the predictions become highly accurate, showing a median error of less than 0.25 seconds. In summary, this article forms a fundamental step towards downstreamed highly accurate position predictions.
An Abstraction-based Method to Check Multi-Agent Deep Reinforcement-Learning Behaviors
Authors: Pierre El Mqirmi, Francesco Belardinelli, Borja G. León
Date: 2021-02-02 00:00:00
Tasks: Deep Reinforcement Learning, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

Multi-agent reinforcement learning (RL) often struggles to ensure the safe behaviours of the learning agents, and therefore it is generally not adapted to safety-critical applications. To address this issue, we present a methodology that combines formal verification with (deep) RL algorithms to guarantee the satisfaction of formally-specified safety constraints both in training and testing. The approach we propose expresses the constraints to verify in Probabilistic Computation Tree Logic (PCTL) and builds an abstract representation of the system to reduce the complexity of the verification step. This abstract model allows for model checking techniques to identify a set of abstract policies that meet the safety constraints expressed in PCTL. Then, the agents' behaviours are restricted according to these safe abstract policies. We provide formal guarantees that by using this method, the actions of the agents always meet the safety constraints, and provide a procedure to generate an abstract model automatically. We empirically evaluate and show the effectiveness of our method in a multi-agent environment.
Android Controlled Mobile Robot Design with IP Camera
Authors: Emre Demir, Ahmet Gokcen, Yakup Kutlu
Date: 2021-01-20 00:00:00
Tasks: Navigate

In this study Arduino card based mobile robot design was realized. This robot can serve as a security robot, an auxiliary robot or a control robot. The designed robot has two operation modes. The first operating mode is autonomous mode. In this mode, the robot detects the surroundings with the help of ultrasonic sensors placed around it, and keeps track of the places it passes by using the encoder. It is able to navigate without hitting any place and passing from where it passes, and it transmits the patient's pulse and temperature condition to the user by other systems installed on it. Also the IP camera sends the scene on the screen. The emergency button to be placed next to the patient sends information to the user in emergency situations. If the abnormality is detected in the temperature and pulse again, the user gives a message. When the pre-recorded drug use times come, the system can alert the patient. The second mode is manual mode. In this mode, the user can move the desired direction of the robot with the Android operating system. In addition, all data received in autonomous mode can be sent to the user. Thus, the user can control the mobile robot with the camera image even if it is not in the vicinity of the robot.
Heterogeneous Graph based Deep Learning for Biomedical Network Link Prediction
Authors: Jinjiang Guo, Jie Li, Dawei Leng, Lurong Pan
Date: 2021-01-28 00:00:00
Tasks: Deep Learning, Link Prediction

Multi-scale biomedical knowledge networks are expanding with emerging experimental technologies that generates multi-scale biomedical big data. Link prediction is increasingly used especially in bipartite biomedical networks to identify hidden biological interactions and relationshipts between key entities such as compounds, targets, gene and diseases. We propose a Graph Neural Networks (GNN) method, namely Graph Pair based Link Prediction model (GPLP), for predicting biomedical network links simply based on their topological interaction information. In GPLP, 1-hop subgraphs extracted from known network interaction matrix is learnt to predict missing links. To evaluate our method, three heterogeneous biomedical networks were used, i.e. Drug-Target Interaction network (DTI), Compound-Protein Interaction network (CPI) from NIH Tox21, and Compound-Virus Inhibition network (CVI). Our proposed GPLP method significantly outperforms over the state-of-the-art baselines. In addition, different network incompleteness is analysed with our devised protocol, and we also design an effective approach to improve the model robustness towards incomplete networks. Our method demonstrates the potential applications in other biomedical networks.
RectiNet-v2: A stacked network architecture for document image dewarping
Authors: Hmrishav Bandyopadhyay, Tanmoy Dasgupta, Nibaran Das, Mita Nasipuri
Date: 2021-02-01 00:00:00
Tasks: Decoder, MS-SSIM, SSIM

With the advent of mobile and hand-held cameras, document images have found their way into almost every domain. Dewarping of these images for the removal of perspective distortions and folds is essential so that they can be understood by document recognition algorithms. For this, we propose an end-to-end CNN architecture that can produce distortion free document images from warped documents it takes as input. We train this model on warped document images simulated synthetically to compensate for lack of enough natural data. Our method is novel in the use of a bifurcated decoder with shared weights to prevent intermingling of grid coordinates, in the use of residual networks in the U-Net skip connections to allow flow of data from different receptive fields in the model, and in the use of a gated network to help the model focus on structure and line level detail of the document image. We evaluate our method on the DocUNet dataset, a benchmark in this domain, and obtain results comparable to state-of-the-art methods.
toon2real: Translating Cartoon Images to Realistic Images
Authors: K. M. Arefeen Sultan, Mohammad Imrul Jubair, Md. Nahidul Islam, Sayed Hossain Khan
Date: 2021-02-01 00:00:00
Tasks: Image-to-Image Translation, Translation

In terms of Image-to-image translation, Generative Adversarial Networks (GANs) has achieved great success even when it is used in the unsupervised dataset. In this work, we aim to translate cartoon images to photo-realistic images using GAN. We apply several state-of-the-art models to perform this task; however, they fail to perform good quality translations. We observe that the shallow difference between these two domains causes this issue. Based on this idea, we propose a method based on CycleGAN model for image translation from cartoon domain to photo-realistic domain. To make our model efficient, we implemented Spectral Normalization which added stability in our model. We demonstrate our experimental results and show that our proposed model has achieved the lowest Frechet Inception Distance score and better results compared to another state-of-the-art technique, UNIT.
Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes
Authors: Keyang Zhou, Bharat Lal Bhatnagar, Bernt Schiele, Gerard Pons-Moll
Date: 2021-02-01 00:00:00
Tasks: Disentanglement

Most learning methods for 3D data (point clouds, meshes) suffer significant performance drops when the data is not carefully aligned to a canonical orientation. Aligning real world 3D data collected from different sources is non-trivial and requires manual intervention. In this paper, we propose the Adjoint Rigid Transform (ART) Network, a neural module which can be integrated with a variety of 3D networks to significantly boost their performance. ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks such as shape reconstruction, interpolation, non-rigid registration, and latent disentanglement. ART achieves this with self-supervision and a rotation equivariance constraint on predicted rotations. The remarkable result is that with only self-supervision, ART facilitates learning a unique canonical orientation for both rigid and nonrigid shapes, which leads to a notable boost in performance of aforementioned tasks. We will release our code and pre-trained models for further research.
Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation
Authors: Peiye Zhuang, Oluwasanmi Koyejo, Alexander G. Schwing
Date: 2021-02-01 00:00:00
Proceeding: ICLR 2021 1
Tasks: Attribute, Image Manipulation

Controllable semantic image editing enables a user to change entire image attributes with a few clicks, e.g., gradually making a summer scene look like it was taken in winter. Classic approaches for this task use a Generative Adversarial Net (GAN) to learn a latent space and suitable latent-space transformations. However, current approaches often suffer from attribute edits that are entangled, global image identity changes, and diminished photo-realism. To address these concerns, we learn multiple attribute transformations simultaneously, integrate attribute regression into the training of transformation functions, and apply a content loss and an adversarial loss that encourages the maintenance of image identity and photo-realism. We propose quantitative evaluation strategies for measuring controllable editing performance, unlike prior work, which primarily focuses on qualitative evaluation. Our model permits better control for both single- and multiple-attribute editing while preserving image identity and realism during transformation. We provide empirical results for both natural and synthetic images, highlighting that our model achieves state-of-the-art performance for targeted image manipulation.
Tight Integration of Feature-based Relocalization in Monocular Direct Visual Odometry
Authors: Mariia Gladkova, Rui Wang, Niclas Zeller, Daniel Cremers
Date: 2021-02-01 00:00:00
Tasks: Visual Odometry

In this paper we propose a framework for integrating map-based relocalization into online direct visual odometry. To achieve map-based relocalization for direct methods, we integrate image features into Direct Sparse Odometry (DSO) and rely on feature matching to associate online visual odometry (VO) with a previously built map. The integration of the relocalization poses is threefold. Firstly, they are incorporated as pose priors in the direct image alignment of the front-end tracking. Secondly, they are tightly integrated into the back-end bundle adjustment. Thirdly, an online fusion module is further proposed to combine relative VO poses and global relocalization poses in a pose graph to estimate keyframe-wise smooth and globally accurate poses. We evaluate our method on two multi-weather datasets showing the benefits of integrating different handcrafted and learned features and demonstrating promising improvements on camera tracking accuracy.
Atlas-aware ConvNetfor Accurate yet Robust Anatomical Segmentation
Authors: Yuan Liang, Weinan Song, Jiawei Yang, Liang Qiu, Kun Wang, Lei He
Date: 2021-02-02 00:00:00

Convolutional networks (ConvNets) have achieved promising accuracy for various anatomical segmentation tasks. Despite the success, these methods can be sensitive to data appearance variations. Considering the large variability of scans caused by artifacts, pathologies, and scanning setups, robust ConvNets are vital for clinical applications, while have not been fully explored. In this paper, we propose to mitigate the challenge by enabling ConvNets' awareness of the underlying anatomical invariances among imaging scans. Specifically, we introduce a fully convolutional Constraint Adoption Module (CAM) that incorporates probabilistic atlas priors as explicit constraints for predictions over a locally connected Conditional Random Field (CFR), which effectively reinforces the anatomical consistency of the labeling outputs. We design the CAM to be flexible for boosting various ConvNet, and compact for co-optimizing with ConvNets for fusion parameters that leads to the optimal performance. We show the advantage of such atlas priors fusion is two-fold with two brain parcellation tasks. First, our models achieve state-of-the-art accuracy among ConvNet-based methods on both datasets, by significantly reducing structural abnormalities of predictions. Second, we can largely boost the robustness of existing ConvNets, proved by: (i) testing on scans with synthetic pathologies, and (ii) training and evaluation on scans of different scanning setups across datasets. Our method is proposing to be easily adopted to existing ConvNets by fine-tuning with CAM plugged in for accuracy and robustness boosts.
Deep Online Fused Video Stabilization
Authors: Zhenmei Shi, Fuhao Shi, Wei-Sheng Lai, Chia-Kai Liang, YIngyu Liang
Date: 2021-02-02 00:00:00
Tasks: Optical Flow Estimation, Video Stabilization

We present a deep neural network (DNN) that uses both sensor data (gyroscope) and image content (optical flow) to stabilize videos through unsupervised learning. The network fuses optical flow with real/virtual camera pose histories into a joint motion representation. Next, the LSTM block infers the new virtual camera pose, and this virtual pose is used to generate a warping grid that stabilizes the frame. Novel relative motion representation as well as a multi-stage training process are presented to optimize our model without any supervision. To the best of our knowledge, this is the first DNN solution that adopts both sensor data and image for stabilization. We validate the proposed framework through ablation studies and demonstrated the proposed method outperforms the state-of-art alternative solutions via quantitative evaluations and a user study.
Progressive Localization Networks for Language-based Moment Localization
Authors: Qi Zheng, Jianfeng Dong, Xiaoye Qu, Xun Yang, Yabing Wang, Pan Zhou, Baolong Liu, Xun Wang
Date: 2021-02-02 00:00:00

This paper targets the task of language-based video moment localization. The language-based setting of this task allows for an open set of target activities, resulting in a large variation of the temporal lengths of video moments. Most existing methods prefer to first sample sufficient candidate moments with various temporal lengths, and then match them with the given query to determine the target moment. However, candidate moments generated with a fixed temporal granularity may be suboptimal to handle the large variation in moment lengths. To this end, we propose a novel multi-stage Progressive Localization Network (PLN) which progressively localizes the target moment in a coarse-to-fine manner. Specifically, each stage of PLN has a localization branch, and focuses on candidate moments that are generated with a specific temporal granularity. The temporal granularities of candidate moments are different across the stages. Moreover, we devise a conditional feature manipulation module and an upsampling connection to bridge the multiple localization branches. In this fashion, the later stages are able to absorb the previously learned information, thus facilitating the more fine-grained localization. Extensive experiments on three public datasets demonstrate the effectiveness of our proposed PLN for language-based moment localization, especially for localizing short moments in long videos.
Single Model Deep Learning on Imbalanced Small Datasets for Skin Lesion Classification
Authors: Peng Yao, Shuwei Shen, Mengjuan Xu, Peng Liu, Fan Zhang, Jinyu Xing, Pengfei Shao, Benjamin Kaffenberger, Ronald X. Xu
Date: 2021-02-02 00:00:00
Tasks: Classification, Data Augmentation, Diagnostic, General Classification, Lesion Classification, Skin Lesion Classification

Deep convolutional neural network (DCNN) models have been widely explored for skin disease diagnosis and some of them have achieved the diagnostic outcomes comparable or even superior to those of dermatologists. However, broad implementation of DCNN in skin disease detection is hindered by small size and data imbalance of the publically accessible skin lesion datasets. This paper proposes a novel single-model based strategy for classification of skin lesions on small and imbalanced datasets. First, various DCNNs are trained on different small and imbalanced datasets to verify that the models with moderate complexity outperform the larger models. Second, regularization DropOut and DropBlock are added to reduce overfitting and a Modified RandAugment augmentation strategy is proposed to deal with the defects of sample underrepresentation in the small dataset. Finally, a novel Multi-Weighted New Loss (MWNL) function and an end-to-end cumulative learning strategy (CLS) are introduced to overcome the challenge of uneven sample size and classification difficulty and to reduce the impact of abnormal samples on training. By combining Modified RandAugment, MWNL and CLS, our single DCNN model method achieved the classification accuracy comparable or superior to those of multiple ensembling models on different dermoscopic image datasets. Our study shows that this method is able to achieve a high classification performance at a low cost of computational resources and inference time, potentially suitable to implement in mobile devices for automated screening of skin lesions and many other malignancies in low resource settings.
GCF-Net: Gated Clip Fusion Network for Video Action Recognition
Authors: Jenhao Hsiao, Jiawei Chen, Chiuman Ho
Date: 2021-02-02 00:00:00
Tasks: Action Recognition, Temporal Action Localization

In recent years, most of the accuracy gains for video action recognition have come from the newly designed CNN architectures (e.g., 3D-CNNs). These models are trained by applying a deep CNN on single clip of fixed temporal length. Since each video segment are processed by the 3D-CNN module separately, the corresponding clip descriptor is local and the inter-clip relationships are inherently implicit. Common method that directly averages the clip-level outputs as a video-level prediction is prone to fail due to the lack of mechanism that can extract and integrate relevant information to represent the video. In this paper, we introduce the Gated Clip Fusion Network (GCF-Net) that can greatly boost the existing video action classifiers with the cost of a tiny computation overhead. The GCF-Net explicitly models the inter-dependencies between video clips to strengthen the receptive field of local clip descriptors. Furthermore, the importance of each clip to an action event is calculated and a relevant subset of clips is selected accordingly for a video-level analysis. On a large benchmark dataset (Kinetics-600), the proposed GCF-Net elevates the accuracy of existing action classifiers by 11.49% (based on central clip) and 3.67% (based on densely sampled clips) respectively.
Learning Crisp Boundaries Using Deep Refinement Network and Adaptive Weighting Loss
Authors: Yi-Jun Cao, Chuan Lin, Yong-Jie Li
Date: 2021-02-02 00:00:00
Tasks: Boundary Detection

Significant progress has been made in boundary detection with the help of convolutional neural networks. Recent boundary detection models not only focus on real object boundary detection but also "crisp" boundaries (precisely localized along the object's contour). There are two methods to evaluate crisp boundary performance. One uses more strict tolerance to measure the distance between the ground truth and the detected contour. The other focuses on evaluating the contour map without any postprocessing. In this study, we analyze both methods and conclude that both methods are two aspects of crisp contour evaluation. Accordingly, we propose a novel network named deep refinement network (DRNet) that stacks multiple refinement modules to achieve richer feature representation and a novel loss function, which combines cross-entropy and dice loss through effective adaptive fusion. Experimental results demonstrated that we achieve state-of-the-art performance for several available datasets.
Tooth Instance Segmentation from Cone-Beam CT Images through Point-based Detection and Gaussian Disentanglement
Authors: Jusang Lee, Minyoung Chung, Minkyung Lee, Yeong-Gil Shin
Date: 2021-02-02 00:00:00
Tasks: Disentanglement, Instance Segmentation, regression, Segmentation, Semantic Segmentation

Individual tooth segmentation and identification from cone-beam computed tomography images are preoperative prerequisites for orthodontic treatments. Instance segmentation methods using convolutional neural networks have demonstrated ground-breaking results on individual tooth segmentation tasks, and are used in various medical imaging applications. While point-based detection networks achieve superior results on dental images, it is still a challenging task to distinguish adjacent teeth because of their similar topologies and proximate nature. In this study, we propose a point-based tooth localization network that effectively disentangles each individual tooth based on a Gaussian disentanglement objective function. The proposed network first performs heatmap regression accompanied by box regression for all the anatomical teeth. A novel Gaussian disentanglement penalty is employed by minimizing the sum of the pixel-wise multiplication of the heatmaps for all adjacent teeth pairs. Subsequently, individual tooth segmentation is performed by converting a pixel-wise labeling task to a distance map regression task to minimize false positives in adjacent regions of the teeth. Experimental results demonstrate that the proposed algorithm outperforms state-of-the-art approaches by increasing the average precision of detection by 9.1%, which results in a high performance in terms of individual tooth segmentation. The primary significance of the proposed method is two-fold: 1) the introduction of a point-based tooth detection framework that does not require additional classification and 2) the design of a novel loss function that effectively separates Gaussian distributions based on heatmap responses in the point-based detection framework.
Enabling energy efficient machine learning on a Ultra-Low-Power vision sensor for IoT
Authors: Francesco Paissan, Massimo Gottardi, Elisabetta Farella
Date: 2021-02-02 00:00:00
Tasks: BIG-bench Machine Learning, Motion Detection

The Internet of Things (IoT) and smart city paradigm includes ubiquitous technology to extract context information in order to return useful services to users and citizens. An essential role in this scenario is often played by computer vision applications, requiring the acquisition of images from specific devices. The need for high-end cameras often penalizes this process since they are power-hungry and ask for high computational resources to be processed. Thus, the availability of novel low-power vision sensors, implementing advanced features like in-hardware motion detection, is crucial for computer vision in the IoT domain. Unfortunately, to be highly energy-efficient, these sensors might worsen the perception performance (e.g., resolution, frame rate, color). Therefore, domain-specific pipelines are usually delivered in order to exploit the full potential of these cameras. This paper presents the development, analysis, and embedded implementation of a realtime detection, classification and tracking pipeline able to exploit the full potential of background filtering Smart Vision Sensors (SVS). The power consumption obtained for the inference - which requires 8ms - is 7.5 mW.