pwc-archive/papers-with-abstracts huggingface dataset:
HR-CAM: Precise Localization of Pathology Using Multi-level Learning in CNNs
Authors: Sumeet Shinde, Tanay Chougule, Jitender Saini, Madhura Ingalhalikar
Date: 2019-09-23 00:00:00

We propose a CNN based technique that aggregates feature maps from its multiple layers that can localize abnormalities with greater details as well as predict pathology under consideration. Existing class activation mapping (CAM) techniques extract feature maps from either the final layer or a single intermediate layer to create the discriminative maps and then interpolate to upsample to the original image resolution. In this case, the subject specific localization is coarse and is unable to capture subtle abnormalities. To mitigate this, our method builds a novel CNN based discriminative localization model that we call high resolution CAM (HR-CAM), which accounts for layers from each resolution, therefore facilitating a comprehensive map that can delineate the pathology for each subject by combining low-level, intermediate as well as high-level features from the CNN. Moreover, our model directly provides the discriminative map in the resolution of the original image facilitating finer delineation of abnormalities. We demonstrate the working of our model on a simulated abnormalities data where we illustrate how the model captures finer details in the final discriminative maps as compared to current techniques. We then apply this technique: (1) to classify ependymomas from grade IV glioblastoma on T1-weighted contrast enhanced (T1-CE) MRI and (2) to predict Parkinson's disease from neuromelanin sensitive MRI. In all these cases we demonstrate that our model not only predicts pathologies with high accuracies, but also creates clinically interpretable subject specific high resolution discriminative localizations. Overall, the technique can be generalized to any CNN and carries high relevance in a clinical setting.
Using machine learning to construct velocity fields from OH-PLIF images
Authors: Shivam Barwey, Malik Hassanaly, Venkat Raman, Adam Steinberg
Date: 2019-09-22 00:00:00
Tasks: BIG-bench Machine Learning, MORPH

This work utilizes data-driven methods to morph a series of time-resolved experimental OH-PLIF images into corresponding three-component planar PIV fields in the closed domain of a premixed swirl combustor. The task is carried out with a fully convolutional network, which is a type of convolutional neural network (CNN) used in many applications in machine learning, alongside an existing experimental dataset which consists of simultaneous OH-PLIF and PIV measurements in both attached and detached flame regimes. Two types of models are compared: 1) a global CNN which is trained using images from the entire domain, and 2) a set of local CNNs, which are trained only on individual sections of the domain. The locally trained models show improvement in creating mappings in the detached regime over the global models. A comparison between model performance in attached and detached regimes shows that the CNNs are much more accurate across the board in creating velocity fields for attached flames. Inclusion of time history in the PLIF input resulted in small noticeable improvement on average, which could imply a greater physical role of instantaneous spatial correlations in the decoding process over temporal dependencies from the perspective of the CNN. Additionally, the performance of local models trained to produce mappings in one section of the domain is tested on other, unexplored sections of the domain. Interestingly, local CNN performance on unseen domain regions revealed the models' ability to utilize symmetry and antisymmetry in the velocity field. Ultimately, this work shows the powerful ability of the CNN to decode the three-dimensional PIV fields from input OH-PLIF images, providing a potential groundwork for a very useful tool for experimental configurations in which accessibility of forms of simultaneous measurements are limited.
Video Skimming: Taxonomy and Comprehensive Survey
Authors: Vivekraj V. K., Debashis Sen, Balasubramanian Raman
Date: 2019-09-21 00:00:00
Tasks: Survey, Video Summarization

Video skimming, also known as dynamic video summarization, generates a temporally abridged version of a given video. Skimming can be achieved by identifying significant components either in uni-modal or multi-modal features extracted from the video. Being dynamic in nature, video skimming, through temporal connectivity, allows better understanding of the video from its summary. Having this obvious advantage, recently, video skimming has drawn the focus of many researchers benefiting from the easy availability of the required computing resources. In this paper, we provide a comprehensive survey on video skimming focusing on the substantial amount of literature from the past decade. We present a taxonomy of video skimming approaches, and discuss their evolution highlighting key advances. We also provide a study on the components required for the evaluation of a video skimming performance.
A Lightweight Deep Learning Model for Human Activity Recognition on Edge Devices
Authors: Preeti Agarwal, Mansaf Alam
Date: 2019-09-20 00:00:00
Tasks: Activity Recognition, Deep Learning, Edge-computing, Human Activity Recognition

Human Activity Recognition (HAR) using wearable and mobile sensors has gained momentum in last few years, in various fields, such as, healthcare, surveillance, education, entertainment. Nowadays, Edge Computing has emerged to reduce communication latency and network traffic.Edge devices are resource constrained devices and cannot support high computation. In literature, various models have been developed for HAR. In recent years, deep learning algorithms have shown high performance in HAR, but these algorithms require lot of computation making them inefficient to be deployed on edge devices. This paper, proposes a Lightweight Deep Learning Model for HAR requiring less computational power, making it suitable to be deployed on edge devices. The performance of proposed model is tested on the participants six daily activities data. Results show that the proposed model outperforms many of the existing machine learning and deep learning techniques.
Interpreting Distortions in Dimensionality Reduction by Superimposing Neighbourhood Graphs
Authors: Benoît Colange, Laurent Vuillon, Sylvain Lespinats, Denys Dutykh
Date: 2019-09-20 00:00:00
Tasks: Dimensionality Reduction

To perform visual data exploration, many dimensionality reduction methods have been developed. These tools allow data analysts to represent multidimensional data in a 2D or 3D space, while preserving as much relevant information as possible. Yet, they cannot preserve all structures simultaneously and they induce some unavoidable distortions. Hence, many criteria have been introduced to evaluate a map's overall quality, mostly based on the preservation of neighbourhoods. Such global indicators are currently used to compare several maps, which helps to choose the most appropriate mapping method and its hyperparameters. However, those aggregated indicators tend to hide the local repartition of distortions. Thereby, they need to be supplemented by local evaluation to ensure correct interpretation of maps. In this paper, we describe a new method, called MING, for `Map Interpretation using Neighbourhood Graphs'. It offers a graphical interpretation of pairs of map quality indicators, as well as local evaluation of the distortions. This is done by displaying on the map the nearest neighbours graphs computed in the data space and in the embedding. Shared and unshared edges exhibit reliable and unreliable neighbourhood information conveyed by the mapping. By this mean, analysts may determine whether proximity (or remoteness) of points on the map faithfully represents similarity (or dissimilarity) of original data, within the meaning of a chosen map quality criteria. We apply this approach to two pairs of widespread indicators: precision/recall and trustworthiness/continuity, chosen for their wide use in the community, which will allow an easy handling by users.
Towards Multimodal Understanding of Passenger-Vehicle Interactions in Autonomous Vehicles: Intent/Slot Recognition Utilizing Audio-Visual Data
Authors: Eda Okur, Shachi H. Kumar, Saurav Sahay, Lama Nachman
Date: 2019-09-20 00:00:00
Tasks: Autonomous Vehicles, Intent Detection, slot-filling, Slot Filling

Understanding passenger intents from spoken interactions and car's vision (both inside and outside the vehicle) are important building blocks towards developing contextual dialog systems for natural interactions in autonomous vehicles (AV). In this study, we continued exploring AMIE (Automated-vehicle Multimodal In-cabin Experience), the in-cabin agent responsible for handling certain multimodal passenger-vehicle interactions. When the passengers give instructions to AMIE, the agent should parse such commands properly considering available three modalities (language/text, audio, video) and trigger the appropriate functionality of the AV system. We had collected a multimodal in-cabin dataset with multi-turn dialogues between the passengers and AMIE using a Wizard-of-Oz scheme via realistic scavenger hunt game. In our previous explorations, we experimented with various RNN-based models to detect utterance-level intents (set destination, change route, go faster, go slower, stop, park, pull over, drop off, open door, and others) along with intent keywords and relevant slots (location, position/direction, object, gesture/gaze, time-guidance, person) associated with the action to be performed in our AV scenarios. In this recent work, we propose to discuss the benefits of multimodal understanding of in-cabin utterances by incorporating verbal/language input (text and speech embeddings) together with the non-verbal/acoustic and visual input from inside and outside the vehicle (i.e., passenger gestures and gaze from in-cabin video stream, referred objects outside of the vehicle from the road view camera stream). Our experimental results outperformed text-only baselines and with multimodality, we achieved improved performances for utterance-level intent detection and slot filling.
Towards Federated Graph Learning for Collaborative Financial Crimes Detection
Authors: Toyotaro Suzumura, Yi Zhou, Natahalie Baracaldo, Guangnan Ye, Keith Houck, Ryo Kawahara, Ali Anwar, Lucia Larise Stavarache, Yuji Watanabe, Pablo Loyola, Daniel Klyashtorny, Heiko Ludwig, Kumar Bhaskaran
Date: 2019-09-19 00:00:00
Tasks: Federated Learning, Graph Learning

Financial crime is a large and growing problem, in some way touching almost every financial institution. Financial institutions are the front line in the war against financial crime and accordingly, must devote substantial human and technology resources to this effort. Current processes to detect financial misconduct have limitations in their ability to effectively differentiate between malicious behavior and ordinary financial activity. These limitations tend to result in gross over-reporting of suspicious activity that necessitate time-intensive and costly manual review. Advances in technology used in this domain, including machine learning based approaches, can improve upon the effectiveness of financial institutions' existing processes, however, a key challenge that most financial institutions continue to face is that they address financial crimes in isolation without any insight from other firms. Where financial institutions address financial crimes through the lens of their own firm, perpetrators may devise sophisticated strategies that may span across institutions and geographies. Financial institutions continue to work relentlessly to advance their capabilities, forming partnerships across institutions to share insights, patterns and capabilities. These public-private partnerships are subject to stringent regulatory and data privacy requirements, thereby making it difficult to rely on traditional technology solutions. In this paper, we propose a methodology to share key information across institutions by using a federated graph learning platform that enables us to build more accurate machine learning models by leveraging federated learning and also graph learning approaches. We demonstrated that our federated model outperforms local model by 20% with the UK FCA TechSprint data set. This new platform opens up a door to efficiently detecting global money laundering activity.
Toward Robust Image Classification
Authors: Basemah Alshemali, Alta Graham, Jugal Kalita
Date: 2019-09-19 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

Neural networks are frequently used for image classification, but can be vulnerable to misclassification caused by adversarial images. Attempts to make neural network image classification more robust have included variations on preprocessing (cropping, applying noise, blurring), adversarial training, and dropout randomization. In this paper, we implemented a model for adversarial detection based on a combination of two of these techniques: dropout randomization with preprocessing applied to images within a given Bayesian uncertainty. We evaluated our model on the MNIST dataset, using adversarial images generated using Fast Gradient Sign Method (FGSM), Jacobian-based Saliency Map Attack (JSMA) and Basic Iterative Method (BIM) attacks. Our model achieved an average adversarial image detection accuracy of 97%, with an average image classification accuracy, after discarding images flagged as adversarial, of 99%. Our average detection accuracy exceeded that of recent papers using similar techniques.
Time Series Modeling for Dream Team in Fantasy Premier League
Authors: Akhil Gupta
Date: 2019-09-19 00:00:00
Tasks: Management, Time Series, Time Series Analysis, Time Series Prediction

The performance of football players in English Premier League varies largely from season to season and for different teams. It is evident that a method capable of forecasting and analyzing the future of these players on-field antics shall assist the management to a great extent. In a simulated environment like the Fantasy Premier League, enthusiasts from all over the world participate and manage the players catalogue for the entire season. Due to the dynamic nature of points system, there is no known approach for the formulation of a dream team. This study aims to tackle this problem by using a hybrid of Autoregressive Integrated Moving Average (ARIMA) and Recurrent Neural Networks (RNNs) for time series prediction of player points and subsequent maximization of total points using Linear Programming (LPP). Given the player points for the past three seasons, the predictions have been made for the current season by modeling differently for ARIMA and RNN, and then creating an ensemble of the same. Prior to that, proper data preprocessing techniques were deployed to enhance the efficacy of the prepared model. Constraints on the type of players like goalkeepers, defenders, midfielders and forwards along with the total budget were effectively optimized using LPP approach. The validation of the proposed team was done with the performance in upcoming season, where the players outperform as expected, and helped in strengthening the feasibility of the solution. Likewise, the proposed approach can be extended to English Premier League by official managers on-field.
Deeply Matting-based Dual Generative Adversarial Network for Image and Document Label Supervision
Authors: Yubao Liu, Kai Lin
Date: 2019-09-19 00:00:00
Tasks: Generative Adversarial Network, Image Matting, Image Super-Resolution, Super-Resolution

Although many methods have been proposed to deal with nature image super-resolution (SR) and get impressive performance, the text images SR is not good due to their ignorance of document images. In this paper, we propose a matting-based dual generative adversarial network (mdGAN) for document image SR. Firstly, the input image is decomposed into document text, foreground and background layers using deep image matting. Then two parallel branches are constructed to recover text boundary information and color information respectively. Furthermore, in order to improve the restoration accuracy of characters in output image, we use the input image's corresponding ground truth text label as extra supervise information to refine the two-branch networks during training. Experiments on real text images demonstrate that our method outperforms several state-of-the-art methods quantitatively and qualitatively.
Responsible Facial Recognition and Beyond
Authors: Yi Zeng, Enmeng Lu, Yinqian Sun, Ruochen Tian
Date: 2019-09-19 00:00:00
Tasks: Gait Recognition, Iris Recognition

Facial recognition is changing the way we live in and interact with our society. Here we discuss the two sides of facial recognition, summarizing potential risks and current concerns. We introduce current policies and regulations in different countries. Very importantly, we point out that the risks and concerns are not only from facial recognition, but also realistically very similar to other biometric recognition technology, including but not limited to gait recognition, iris recognition, fingerprint recognition, voice recognition, etc. To create a responsible future, we discuss possible technological moves and efforts that should be made to keep facial recognition (and biometric recognition in general) developing for social good.
EPOSIT: An Absolute Pose Estimation Method for Pinhole and Fish-Eye Cameras
Authors: Zhaobing Kang, Wei Zou, Zheng Zhu, Chi Zhang, Hongxuan Ma
Date: 2019-09-19 00:00:00
Tasks: Camera Pose Estimation, Pose Estimation

This paper presents a generic 6DOF camera pose estimation method, which can be used for both the pinhole camera and the fish-eye camera. Different from existing methods, relative positions of 3D points rather than absolute coordinates in the world coordinate system are employed in our method, and it has a unique solution. The application scope of POSIT (Pose from Orthography and Scaling with Iteration) algorithm is generalized to fish-eye cameras by combining with the radially symmetric projection model. The image point relationship between the pinhole camera and the fish-eye camera is derived based on their projection model. The general pose expression which fits for different cameras can be acquired by four noncoplanar object points and their corresponding image points. Accurate estimation results are calculated iteratively. Experimental results on synthetic and real data show that the pose estimation results of our method are more stable and accurate than state-of-the-art methods. The source code is available at https://github.com/k032131/EPOSIT.
Graph Neural Networks for Image Understanding Based on Multiple Cues: Group Emotion Recognition and Event Recognition as Use Cases
Authors: Xin Guo, Luisa F. Polania, Bin Zhu, Charles Boncelet, Kenneth E. Barner
Date: 2019-09-19 00:00:00
Tasks: Emotion Recognition, Graph Neural Network

A graph neural network (GNN) for image understanding based on multiple cues is proposed in this paper. Compared to traditional feature and decision fusion approaches that neglect the fact that features can interact and exchange information, the proposed GNN is able to pass information among features extracted from different models. Two image understanding tasks, namely group-level emotion recognition (GER) and event recognition, which are highly semantic and require the interaction of several deep models to synthesize multiple cues, were selected to validate the performance of the proposed method. It is shown through experiments that the proposed method achieves state-of-the-art performance on the selected image understanding tasks. In addition, a new group-level emotion recognition database is introduced and shared in this paper.
Unsupervised Segmentation of Fire and Smoke from Infra-Red Videos
Authors: Meenu Ajith, Manel Martínez-Ramón
Date: 2019-09-18 00:00:00
Tasks: Clustering, Fire Detection, Optical Flow Estimation, Segmentation

This paper proposes a vision-based fire and smoke segmentation system which use spatial, temporal and motion information to extract the desired regions from the video frames. The fusion of information is done using multiple features such as optical flow, divergence and intensity values. These features extracted from the images are used to segment the pixels into different classes in an unsupervised way. A comparative analysis is done by using multiple clustering algorithms for segmentation. Here the Markov Random Field performs more accurately than other segmentation algorithms since it characterizes the spatial interactions of pixels using a finite number of parameters. It builds a probabilistic image model that selects the most likely labeling using the maximum a posteriori (MAP) estimation. This unsupervised approach is tested on various images and achieves a frame-wise fire detection rate of 95.39%. Hence this method can be used for early detection of fire in real-time and it can be incorporated into an indoor or outdoor surveillance system.
Student Engagement Detection Using Emotion Analysis, Eye Tracking and Head Movement with Machine Learning
Authors: Prabin Sharma, Shubham Joshi, Subash Gautam, Sneha Maharjan, Salik Ram Khanal, Manuel Cabral Reis, João Barroso, Vítor Manuel de Jesus Filipe
Date: 2019-09-18 00:00:00
Tasks: BIG-bench Machine Learning, Emotion Recognition

With the increase of distance learning, in general, and e-learning, in particular, having a system capable of determining the engagement of students is of primordial importance, and one of the biggest challenges, both for teachers, researchers and policy makers. Here, we present a system to detect the engagement level of the students. It uses only information provided by the typical built-in web-camera present in a laptop computer, and was designed to work in real time. We combine information about the movements of the eyes and head, and facial emotions to produce a concentration index with three classes of engagement: "very engaged", "nominally engaged" and "not engaged at all". The system was tested in a typical e-learning scenario, and the results show that it correctly identifies each period of time where students were "very engaged", "nominally engaged" and "not engaged at all". Additionally, the results also show that the students with best scores also have higher concentration indexes.
The impact of patient clinical information on automated skin cancer detection
Authors: Andre G. C. Pacheco, Renato A. Krohling
Date: 2019-09-16 00:00:00

Skin cancer is one of the most common types of cancer around the world. For this reason, over the past years, different approaches have been proposed to assist detect it. Nonetheless, most of them are based only on dermoscopy images and do not take into account the patient clinical information. In this work, first, we present a new dataset that contains clinical images, acquired from smartphones, and patient clinical information of the skin lesions. Next, we introduce a straightforward approach to combine the clinical data and the images using different well-known deep learning models. These models are applied to the presented dataset using only the images and combining them with the patient clinical information. We present a comprehensive study to show the impact of the clinical data on the final predictions. The results obtained by combining both sets of information show a general improvement of around 7% in the balanced accuracy for all models. In addition, the statistical test indicates significant differences between the models with and without considering both data. The improvement achieved shows the potential of using patient clinical information in skin cancer detection and indicates that this piece of information is important to leverage skin cancer detection systems.