pwc-archive/papers-with-abstracts huggingface dataset:
Forecasting future action sequences with attention: a new approach to weakly supervised action forecasting
Authors: Yan Bin Ng, Basura Fernando
Date: 2019-12-10 00:00:00
Tasks: Machine Translation

Future human action forecasting from partial observations of activities is an important problem in many practical applications such as assistive robotics, video surveillance and security. We present a method to forecast actions for the unseen future of the video using a neural machine translation technique that uses encoder-decoder architecture. The input to this model is the observed RGB video, and the objective is to forecast the correct future symbolic action sequence. Unlike prior methods that make action predictions for some unseen percentage of video one for each frame, we predict the complete action sequence that is required to accomplish the activity. We coin this task action sequence forecasting. To cater for two types of uncertainty in the future predictions, we propose a novel loss function. We show a combination of optimal transport and future uncertainty losses help to improve results. We extend our action sequence forecasting model to perform weakly supervised action forecasting on two challenging datasets, the Breakfast and the 50Salads. Specifically, we propose a model to predict actions of future unseen frames without using frame level annotations during training. Using Fisher vector features, our supervised model outperforms the state-of-the-art action forecasting model by 0.83% and 7.09% on the Breakfast and the 50Salads datasets respectively. Our weakly supervised model is only 0.6% behind the most recent state-of-the-art supervised model and obtains comparable results to other published fully supervised methods, and sometimes even outperforms them on the Breakfast dataset. Most interestingly, our weakly supervised model outperforms prior models by 1.04% leveraging on proposed weakly supervised architecture, and effective use of attention mechanism and loss functions.
Efficient and Robust Reinforcement Learning with Uncertainty-based Value Expansion
Authors: Bo Zhou, Hongsheng Zeng, Fan Wang, Yunxiang Li, Hao Tian
Date: 2019-12-10 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

By integrating dynamics models into model-free reinforcement learning (RL) methods, model-based value expansion (MVE) algorithms have shown a significant advantage in sample efficiency as well as value estimation. However, these methods suffer from higher function approximation errors than model-free methods in stochastic environments due to a lack of modeling the environmental randomness. As a result, their performance lags behind the best model-free algorithms in some challenging scenarios. In this paper, we propose a novel Hybrid-RL method that builds on MVE, namely the Risk Averse Value Expansion (RAVE). With imaginative rollouts generated by an ensemble of probabilistic dynamics models, we further introduce the aversion of risks by seeking the lower confidence bound of the estimation. Experiments on a range of challenging environments show that by modeling the uncertainty completely, RAVE substantially enhances the robustness of previous model-based methods, and yields state-of-the-art performance. With this technique, our solution gets the first place in NeurIPS 2019: Learn to Move.
Severity Detection Tool for Patients with Infectious Disease
Authors: Girmaw Abebe Tadesse, Tingting Zhu, Nhan Le Nguyen Thanh, Nguyen Thanh Hung, Ha Thi Hai Duong, Truong Huu Khanh, Pham Van Quang, Duc Duong Tran, LamMinh Yen, H Rogier Van Doorn, Nguyen Van Hao, John Prince, Hamza Javed, DaniKiyasseh, Le Van Tan, Louise Thwaites, David A. Clifton
Date: 2019-12-10 00:00:00
Tasks: Heart Rate Variability

Hand, foot and mouth disease (HFMD) and tetanus are serious infectious diseases in low and middle income countries. Tetanus in particular has a high mortality rate and its treatment is resource-demanding. Furthermore, HFMD often affects a large number of infants and young children. As a result, its treatment consumes enormous healthcare resources, especially when outbreaks occur. Autonomic nervous system dysfunction (ANSD) is the main cause of death for both HFMD and tetanus patients. However, early detection of ANSD is a difficult and challenging problem. In this paper, we aim to provide a proof-of-principle to detect the ANSD level automatically by applying machine learning techniques to physiological patient data, such as electrocardiogram (ECG) and photoplethysmogram (PPG) waveforms, which can be collected using low-cost wearable sensors. Efficient features are extracted that encode variations in the waveforms in the time and frequency domains. A support vector machine is employed to classify the ANSD levels. The proposed approach is validated on multiple datasets of HFMD and tetanus patients in Vietnam. Results show that encouraging performance is achieved in classifying ANSD levels. Moreover, the proposed features are simple, more generalisable and outperformed the standard heart rate variability (HRV) analysis. The proposed approach would facilitate both the diagnosis and treatment of infectious diseases in low and middle income countries, and thereby improve overall patient care.
Graph Markov Network for Traffic Forecasting with Missing Data
Authors: Zhiyong Cui, Longfei Lin, Ziyuan Pu, Yinhai Wang
Date: 2019-12-10 00:00:00
Tasks: Edge-computing, Imputation, Management, Missing Values

Traffic forecasting is a classical task for traffic management and it plays an important role in intelligent transportation systems. However, since traffic data are mostly collected by traffic sensors or probe vehicles, sensor failures and the lack of probe vehicles will inevitably result in missing values in the collected raw data for some specific links in the traffic network. Although missing values can be imputed, existing data imputation methods normally need long-term historical traffic state data. As for short-term traffic forecasting, especially under edge computing and online prediction scenarios, traffic forecasting models with the capability of handling missing values are needed. In this study, we consider the traffic network as a graph and define the transition between network-wide traffic states at consecutive time steps as a graph Markov process. In this way, missing traffic states can be inferred step by step and the spatial-temporal relationships among the roadway links can be Incorporated. Based on the graph Markov process, we propose a new neural network architecture for spatial-temporal data forecasting, i.e. the graph Markov network (GMN). By incorporating the spectral graph convolution operation, we also propose a spectral graph Markov network (SGMN). The proposed models are compared with baseline models and tested on three real-world traffic state datasets with various missing rates. Experimental results show that the proposed GMN and SGMN can achieve superior prediction performance in terms of both accuracy and efficiency. Besides, the proposed models' parameters, weights, and predicted results are comprehensively analyzed and visualized.
Neural Voxel Renderer: Learning an Accurate and Controllable Rendering Tool
Authors: Konstantinos Rematas, Vittorio Ferrari
Date: 2019-12-10 00:00:00
Proceeding: CVPR 2020 6
Tasks: Image Generation, Neural Rendering

We present a neural rendering framework that maps a voxelized scene into a high quality image. Highly-textured objects and scene element interactions are realistically rendered by our method, despite having a rough representation as an input. Moreover, our approach allows controllable rendering: geometric and appearance modifications in the input are accurately propagated to the output. The user can move, rotate and scale an object, change its appearance and texture or modify the position of the light and all these edits are represented in the final rendering. We demonstrate the effectiveness of our approach by rendering scenes with varying appearance, from single color per object to complex, high-frequency textures. We show that our rerendering network can generate very detailed images that represent precisely the appearance of the input scene. Our experiments illustrate that our approach achieves more accurate image synthesis results compared to alternatives and can also handle low voxel grid resolutions. Finally, we show how our neural rendering framework can capture and faithfully render objects from real images and from a diverse set of classes.
FootAndBall: Integrated player and ball detector
Authors: Jacek Komorowski, Grzegorz Kurzejamski, Grzegorz Sarwas
Date: 2019-12-10 00:00:00

The paper describes a deep neural network-based detector dedicated for ball and players detection in high resolution, long shot, video recordings of soccer matches. The detector, dubbed FootAndBall, has an efficient fully convolutional architecture and can operate on input video stream with an arbitrary resolution. It produces ball confidence map encoding the position of the detected ball, player confidence map and player bounding boxes tensor encoding players' positions and bounding boxes. The network uses Feature Pyramid Network desing pattern, where lower level features with higher spatial resolution are combined with higher level features with bigger receptive field. This improves discriminability of small objects (the ball) as larger visual context around the object of interest is taken into account for the classification. Due to its specialized design, the network has two orders of magnitude less parameters than a generic deep neural network-based object detector, such as SSD or YOLO. This allows real-time processing of high resolution input video stream. Our code and pre-trained model can be found on the project website: https://github.com/jac99/FootAndBall .
Modelling curvature of a bent paper leaf
Authors: Sasikanth Raghava Goteti
Date: 2019-12-10 00:00:00
Tasks: BIG-bench Machine Learning

In this article, we briefly describe various tools and approaches that algebraic geometry has to offer to straighten bent objects. Throughout this article we will consider a specific example of a bent or curved piece of paper which in our case acts very much like an elastica curve. We conclude this article with a suggestion to algebraic geometry as a viable and fast performance alternative of neural networks in vision and machine learning. The purpose of this article is not to build a full blown framework but to show possibility of using algebraic geometry as an alternative to neural networks for recognizing or extracting features on manifolds.
Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagation
Authors: Gedas Bertasius, Lorenzo Torresani
Date: 2019-12-10 00:00:00
Proceeding: CVPR 2020 6
Tasks: Instance Segmentation, Object, Segmentation, Semantic Segmentation, Video Instance Segmentation

We introduce a method for simultaneously classifying, segmenting and tracking object instances in a video sequence. Our method, named MaskProp, adapts the popular Mask R-CNN to video by adding a mask propagation branch that propagates frame-level object instance masks from each video frame to all the other frames in a video clip. This allows our system to predict clip-level instance tracks with respect to the object instances segmented in the middle frame of the clip. Clip-level instance tracks generated densely for each frame in the sequence are finally aggregated to produce video-level object instance segmentation and classification. Our experiments demonstrate that our clip-level instance segmentation makes our approach robust to motion blur and object occlusions in video. MaskProp achieves the best reported accuracy on the YouTube-VIS dataset, outperforming the ICCV 2019 video instance segmentation challenge winner despite being much simpler and using orders of magnitude less labeled data (1.3M vs 1B images and 860K vs 14M bounding boxes).
MaskAAE: Latent space optimization for Adversarial Auto-Encoders
Authors: Arnab Kumar Mondal, Sankalan Pal Chowdhury, Aravind Jayendran, Parag Singla, Himanshu Asnani, Prathosh AP
Date: 2019-12-10 00:00:00

The field of neural generative models is dominated by the highly successful Generative Adversarial Networks (GANs) despite their challenges, such as training instability and mode collapse. Auto-Encoders (AE) with regularized latent space provide an alternative framework for generative models, albeit their performance levels have not reached that of GANs. In this work, we hypothesise that the dimensionality of the AE model's latent space has a critical effect on the quality of generated data. Under the assumption that nature generates data by sampling from a "true" generative latent space followed by a deterministic function, we show that the optimal performance is obtained when the dimensionality of the latent space of the AE-model matches with that of the "true" generative latent space. Further, we propose an algorithm called the Mask Adversarial Auto-Encoder (MaskAAE), in which the dimensionality of the latent space of an adversarial auto encoder is brought closer to that of the "true" generative latent space, via a procedure to mask the spurious latent dimensions. We demonstrate through experiments on synthetic and several real-world datasets that the proposed formulation yields betterment in the generation quality.
Understanding 3D CNN Behavior for Alzheimer's Disease Diagnosis from Brain PET Scan
Authors: Jyoti Islam, Yanqing Zhang
Date: 2019-12-10 00:00:00
Tasks: Medical Image Analysis

In recent days, Convolutional Neural Networks (CNN) have demonstrated impressive performance in medical image analysis. However, there is a lack of clear understanding of why and how the Convolutional Neural Network performs so well for image analysis task. How CNN analyzes an image and discriminates among samples of different classes are usually considered as non-transparent. As a result, it becomes difficult to apply CNN based approaches in clinical procedures and automated disease diagnosis systems. In this paper, we consider this issue and work on visualizing and understanding the decision of Convolutional Neural Network for Alzheimer's Disease (AD) Diagnosis. We develop a 3D deep convolutional neural network for AD diagnosis using brain PET scans and propose using five visualizations techniques - Sensitivity Analysis (Backpropagation), Guided Backpropagation, Occlusion, Brain Area Occlusion, and Layer-wise Relevance Propagation (LRP) to understand the decision of the CNN by highlighting the relevant areas in the PET data.
A Feasible Framework for Arbitrary-Shaped Scene Text Recognition
Authors: Jinjin Zhang, Wei Wang, Di Huang, Qingjie Liu, Yunhong Wang
Date: 2019-12-10 00:00:00
Tasks: Instance Segmentation, Language Modeling, Language Modelling, Scene Text Recognition, Semantic Segmentation, Text Detection, Text Spotting

Deep learning based methods have achieved surprising progress in Scene Text Recognition (STR), one of classic problems in computer vision. In this paper, we propose a feasible framework for multi-lingual arbitrary-shaped STR, including instance segmentation based text detection and language model based attention mechanism for text recognition. Our STR algorithm not only recognizes Latin and Non-Latin characters, but also supports arbitrary-shaped text recognition. Our method wins the championship on Scene Text Spotting Task (Latin Only, Latin and Chinese) of ICDAR2019 Robust Reading Challenge on ArbitraryShaped Text Competition. Code is available at https://github.com/zhang0jhon/AttentionOCR.
Accurate Entrance Position Detection Based on Wi-Fi and GPS Signals Using Machine Learning
Authors: Ahmad Abadleh
Date: 2019-12-10 00:00:00
Tasks: BIG-bench Machine Learning, Position

This paper aims at detecting an accurate position of the main entrance of the buildings. The proposed approach relies on the fact that the GPS signals drop significantly when the user enters a building. Moreover, as most of the public buildings provide Wi-Fi services, the Wi-Fi received signal strength (RSS) can be utilized in order to detect the entrance of the buildings. The rationale behind this paper is that the GPS signals decrease as the user gets close to the main entrance and the Wi-Fi signal increases as the user approaches the main entrance. Several real experiments have been conducted in order to guarantee the feasibility of the proposed approach. The experiment results have shown an interesting result and the accuracy of the whole system was one meter
SG-VAE: Scene Grammar Variational Autoencoder to generate new indoor scenes
Authors: Pulak Purkait, Christopher Zach, Ian Reid
Date: 2019-12-10 00:00:00
Proceeding: ECCV 2020 8
Tasks: valid

Deep generative models have been used in recent years to learn coherent latent representations in order to synthesize high-quality images. In this work, we propose a neural network to learn a generative model for sampling consistent indoor scene layouts. Our method learns the co-occurrences, and appearance parameters such as shape and pose, for different objects categories through a grammar-based auto-encoder, resulting in a compact and accurate representation for scene layouts. In contrast to existing grammar-based methods with a user-specified grammar, we construct the grammar automatically by extracting a set of production rules on reasoning about object co-occurrences in training data. The extracted grammar is able to represent a scene by an augmented parse tree. The proposed auto-encoder encodes these parse trees to a latent code, and decodes the latent code to a parse tree, thereby ensuring the generated scene is always valid. We experimentally demonstrate that the proposed auto-encoder learns not only to generate valid scenes (i.e. the arrangements and appearances of objects), but it also learns coherent latent representations where nearby latent samples decode to similar scene outputs. The obtained generative model is applicable to several computer vision tasks such as 3D pose and layout estimation from RGB-D data.
Expansion of Cyber Attack Data From Unbalanced Datasets Using Generative Techniques
Authors: Ibrahim Yilmaz, Rahat Masum
Date: 2019-12-10 00:00:00

Machine learning techniques help to understand patterns of a dataset to create a defense mechanism against cyber attacks. However, it is difficult to construct a theoretical model due to the imbalances in the dataset for discriminating attacks from the overall dataset. Multilayer Perceptron (MLP) technique will provide improvement in accuracy and increase the performance of detecting the attack and benign data from a balanced dataset. We have worked on the UGR'16 dataset publicly available for this work. Data wrangling has been done due to prepare test set from in the original set. We fed the neural network classifier larger input to the neural network in an increasing manner (i.e. 10000, 50000, 1 million) to see the distribution of features over the accuracy. We have implemented a GAN model that can produce samples of different attack labels (e.g. blacklist, anomaly spam, ssh scan). We have been able to generate as many samples as necessary based on the data sample we have taken from the UGR'16. We have tested the accuracy of our model with the imbalance dataset initially and then with the increasing the attack samples and found improvement of classification performance for the latter.
Appending Adversarial Frames for Universal Video Attack
Authors: Zhikai Chen, Lingxi Xie, Shanmin Pang, Yong He, Qi Tian
Date: 2019-12-10 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Video Classification

There have been many efforts in attacking image classification models with adversarial perturbations, but the same topic on video classification has not yet been thoroughly studied. This paper presents a novel idea of video-based attack, which appends a few dummy frames (e.g., containing the texts of `thanks for watching') to a video clip and then adds adversarial perturbations only on these new frames. Our approach enjoys three major benefits, namely, a high success rate, a low perceptibility, and a strong ability in transferring across different networks. These benefits mostly come from the common dummy frame which pushes all samples towards the boundary of classification. On the other hand, such attacks are easily to be concealed since most people would not notice the abnormality behind the perturbed video clips. We perform experiments on two popular datasets with six state-of-the-art video classification models, and demonstrate the effectiveness of our approach in the scenario of universal video attacks.
Automatic Analysis System of Calcaneus Radiograph: Rotation-Invariant Landmark Detection for Calcaneal Angle Measurement, Fracture Identification and Fracture Region Segmentation
Authors: Jia Guo, Yuxuan Mu, Dong Xue, Huiqi Li, Junxian Chen, Huanxin Yan, Hailin Xu, Wei Wang
Date: 2019-12-10 00:00:00
Tasks: Multi-Task Learning, Segmentation

Calcaneus is the largest tarsal bone to withstand the daily stresses of weight-bearing. The calcaneal fracture is the most common type in the tarsal bone fractures. After a fracture is suspected, plain radiographs should be taken first. Bohler's Angle (BA) and Critical Angle of Gissane (CAG), measured by four anatomic landmarks in lateral foot radiograph, can guide fracture diagnosis and facilitate operative recovery of the fractured calcaneus. This study aims to develop an analysis system that can automatically locate four anatomic landmarks, measure BA and CAG for fracture assessment, identify fractured calcaneus, and segment fractured regions. For landmark detection, we proposed a coarse-to-fine Rotation-Invariant Regression-Voting (RIRV) landmark detection method based on regressive Multi-Layer Perceptron (MLP) and Scale Invariant Feature Transform (SIFT) patch descriptor, which solves the problem of fickle rotation of calcaneus. By implementing a novel normalization approach, the RIRV method is explicitly rotation-invariance comparing with traditional regressive methods. For fracture identification and segmentation, a convolution neural network (CNN) based on U-Net with auxiliary classification head (U-Net-CH) is designed. The input ROIs of the CNN are normalized by detected landmarks to uniform view, orientation, and scale. The advantage of this approach is the multi-task learning that combines classification and segmentation. Our system can accurately measure BA and CAG with a mean angle error of 3.8 and 6.2 respectively. For fracture identification and fracture region segmentation, our system presents good performance with an F1-score of 96.55%, recall of 94.99%, and segmentation IoU-score of 0.586.