pwc-archive/papers-with-abstracts huggingface dataset:
Deep User Identification Model with Multiple Biometrics
Authors: Hyoung-Kyu Song, Ebrahim AlAlkeem, Jaewoong Yun, Tae-Ho Kim, Hyerin Yoo, Dasom Heo, Chan Yeob Yeun, Myungsu Chae
Date: 2019-09-03 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Gender Classification, General Classification, model, User Identification

Identification using biometrics is an important yet challenging task. Abundant research has been conducted on identifying personal identity or gender using given signals. Various types of biometrics such as electrocardiogram (ECG), electroencephalogram (EEG), face, fingerprint, and voice have been used for these tasks. Most research has only focused on single modality or a single task, while the combination of input modality or tasks is yet to be investigated. In this paper, we propose deep identification and gender classification using multimodal biometrics. Our model uses ECG, fingerprint, and facial data. It then performs two tasks: gender identification and classification. By engaging multi-modality, a single model can handle various input domains without training each modality independently, and the correlation between domains can increase its generalization performance on the tasks.
Automated Detecting and Placing Road Objects from Street-level Images
Authors: Chaoquan Zhang, Hongchao Fan, Wanzhi Li, Bo Mao, Xuan Ding
Date: 2019-08-29 00:00:00
Tasks: Autonomous Vehicles

Navigation services utilized by autonomous vehicles or ordinary users require the availability of detailed information about road-related objects and their geolocations, especially at road intersections. However, these road intersections are mainly represented as point elements without detailed information, or are even not available in current versions of crowdsourced mapping databases including OpenStreetMap(OSM). This study develops an approach to automatically detect road objects and place them to right location from street-level images. Our processing pipeline relies on two convolutional neural networks: the first segments the images, while the second detects and classifies the specific objects. Moreover, to locate the detected objects, we establish an attributed topological binary tree(ATBT) based on urban grammar for each image to depict the coherent relations of topologies, attributes and semantics of the road objects. Then the ATBT is further matched with map features on OSM to determine the right placed location. The proposed method has been applied to a case study in Berlin, Germany. We validate the effectiveness of our method on two object classes: traffic signs and traffic lights. Experimental results demonstrate that the proposed approach provides near-precise localization results in terms of completeness and positional accuracy. Among many potential applications, the output may be combined with other sources of data to guide autonomous vehicles
An Objectness Score for Accurate and Fast Detection during Navigation
Authors: Hongsun Choi, Mincheul Kang, Youngsun Kwon, Sung-Eui Yoon
Date: 2019-08-26 00:00:00
Tasks: Robot Navigation

We propose a novel method utilizing an objectness score for maintaining the locations and classes of objects detected from Mask R-CNN during mobile robot navigation. The objectness score is defined to measure how well the detector identifies the locations and classes of objects during navigation. Specifically, it is designed to increase when there is sufficient distance between a detected object and the camera. During the navigation process, we transform the locations of objects in 3D world coordinates into 2D image coordinates through an affine projection and decide whether to retain the classes of detected objects using the objectness score. We conducted experiments to determine how well the locations and classes of detected objects are maintained at various angles and positions. Experimental results showed that our approach is efficient and robust, regardless of changing angles and distances.
Reusing Convolutional Activations from Frame to Frame to Speed up Training and Inference
Authors: Arno Khachatourian
Date: 2019-09-02 00:00:00
Tasks: Atari Games, Time Series, Time Series Analysis

When processing similar frames in succession, we can take advantage of the locality of the convolution operation to reevaluate only portions of the image that changed from the previous frame. By saving the output of a layer of convolutions and calculating the change from frame to frame, we can reuse previous activations and save computational resources that would otherwise be wasted recalculating convolutions whose outputs we have already observed. This technique can be applied to many domains, such as processing videos from stationary video cameras, studying the effects of occluding or distorting sections of images, applying convolution to multiple frames of audio or time series data, or playing Atari games. Furthermore, this technique can be applied to speed up both training and inference.
Illuminated Decision Trees with Lucid
Authors: David Mott, Richard Tomsett
Date: 2019-09-03 00:00:00
Tasks: image-classification, Image Classification

The Lucid methods described by Olah et al. (2018) provide a way to inspect the inner workings of neural networks trained on image classification tasks using feature visualization. Such methods have generally been applied to networks trained on visually rich, large-scale image datasets like ImageNet, which enables them to produce enticing feature visualizations. To investigate these methods further, we applied them to classifiers trained to perform the much simpler (in terms of dataset size and visual richness), yet challenging task of distinguishing between different kinds of white blood cell from microscope images. Such a task makes generating useful feature visualizations difficult, as the discriminative features are inherently hard to identify and interpret. We address this by presenting the "Illuminated Decision Tree" approach, in which we use a neural network trained on the task as a feature extractor, then learn a decision tree based on these features, and provide Lucid visualizations for each node in the tree. We demonstrate our approach with several examples, showing how this approach could be useful both in model development and debugging, and when explaining model outputs to non-experts.
PRSNet: Part Relation and Selection Network for Bone Age Assessment
Authors: Yuanfeng Ji, Hao Chen, Dan Lin, Xiaohua Wu, Di Lin
Date: 2019-09-05 00:00:00
Tasks: Relation

Bone age is one of the most important indicators for assessing bone's maturity, which can help to interpret human's growth development level and potential progress. In the clinical practice, bone age assessment (BAA) of X-ray images requires the joint consideration of the appearance and location information of hand bones. These kinds of information can be effectively captured by the relation of different anatomical parts of hand bone. Recently developed methods differ mostly in how they model the part relation and choose useful parts for BAA. However, these methods neglect the mining of relationship among different parts, which can help to improve the assessment accuracy. In this paper, we propose a novel part relation module, which accurately discovers the underlying concurrency of parts by using multi-scale context information of deep learning feature representation. Furthermore, based on the part relation, we explore a new part selection module, which comprehensively measures the importance of parts and select the top ranking parts for assisting BAA. We jointly train our part relation and selection modules in an end-to-end way, achieving state-of-the-art performance on the public RSNA 2017 Pediatric Bone Age benchmark dataset and outperforming other competitive methods by a significant margin.
A Novel Design of Adaptive and Hierarchical Convolutional Neural Networks using Partial Reconfiguration on FPGA
Authors: Mohammad Farhadi, Mehdi Ghasemi, Yezhou Yang
Date: 2019-09-05 00:00:00
Tasks: CPU, Decision Making

Nowadays most research in visual recognition using Convolutional Neural Networks (CNNs) follows the "deeper model with deeper confidence" belief to gain a higher recognition accuracy. At the same time, deeper model brings heavier computation. On the other hand, for a large chunk of recognition challenges, a system can classify images correctly using simple models or so-called shallow networks. Moreover, the implementation of CNNs faces with the size, weight, and energy constraints on the embedded devices. In this paper, we implement the adaptive switching between shallow and deep networks to reach the highest throughput on a resource-constrained MPSoC with CPU and FPGA. To this end, we develop and present a novel architecture for the CNNs where a gate makes the decision whether using the deeper model is beneficial or not. Due to resource limitation on FPGA, the idea of partial reconfiguration has been used to accommodate deep CNNs on the FPGA resources. We report experimental results on CIFAR-10, CIFAR-100, and SVHN datasets to validate our approach. Using confidence metric as the decision making factor, only 69.8%, 71.8%, and 43.8% of the computation in the deepest network is done for CIFAR-10, CIFAR-100, and SVHN while it can maintain the desired accuracy with the throughput of around 400 images per second for SVHN dataset.
What can computational models learn from human selective attention? A review from an audiovisual crossmodal perspective
Authors: Di Fu, Cornelius Weber, Guochun Yang, Matthias Kerzel, Weizhi Nan, Pablo Barros, Haiyan Wu, Xun Liu, Stefan Wermter
Date: 2019-09-05 00:00:00

Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments and computational modeling. Although an increasing number of findings on crossmodal selective attention have shed light on humans' behavioral patterns and neural underpinnings, a much better understanding is still necessary to yield the same benefit for computational intelligent agents. This article reviews studies of selective attention in unimodal visual and auditory and crossmodal audiovisual setups from the multidisciplinary perspectives of psychology and cognitive neuroscience, and evaluates different ways to simulate analogous mechanisms in computational models and robotics. We discuss the gaps between these fields in this interdisciplinary review and provide insights about how to use psychological findings and theories in artificial intelligence from different perspectives.
Assessment of Shift-Invariant CNN Gaze Mappings for PS-OG Eye Movement Sensors
Authors: Henry K. Griffith, Dmytro Katrychuk, Oleg V. Komogortsev
Date: 2019-09-04 00:00:00

Photosensor oculography (PS-OG) eye movement sensors offer desirable performance characteristics for integration within wireless head mounted devices (HMDs), including low power consumption and high sampling rates. To address the known performance degradation of these sensors due to HMD shifts, various machine learning techniques have been proposed for mapping sensor outputs to gaze location. This paper advances the understanding of a recently introduced convolutional neural network designed to provide shift invariant gaze mapping within a specified range of sensor translations. Performance is assessed for shift training examples which better reflect the distribution of values that would be generated through manual repositioning of the HMD during a dedicated collection of training data. The network is shown to exhibit comparable accuracy for this realistic shift distribution versus a previously considered rectangular grid, thereby enhancing the feasibility of in-field set-up. In addition, this work further demonstrates the practical viability of the proposed initialization process by demonstrating robust mapping performance versus training data scale. The ability to maintain reasonable accuracy for shifts extending beyond those introduced during training is also demonstrated.
On-demand teleradiology using smartphone photographs as proxies for DICOM images
Authors: Christine Podilchuk, Siddhartha Pachhai, Robert Warfsman, Richard Mammone
Date: 2019-09-06 00:00:00

The use of photographs of the screen of displayed medical images is explored to circumvent the challenges involved in transferring images between sites. The photographs can be conveniently taken with a smartphone and analyzed remotely by either human or AI experts. An autoencoder preprocessor is shown to improve the performance for human experts. The AI performance provided by photographs is shown to be statistically equivalent to using the original DICOM images. The autoencoder preprocessor increases the PSNR by 15 dB or greater and provides an AUC that is statistically equivalent to using the original DICOM images. The photo approach is an alternative to IHE-based teleradiology applications while avoiding the problems inherit in navigating the proprietary and security barriers that limit DICOM communication between PACS in practice.
Activity recognition using ST-GCN with 3D motion data
Authors: Xin Cao, Wataru Kudo, Chihiro Ito, Masaki Shuzo, Eisaku Maeda
Date: 2019-09-13 00:00:00
Proceeding: UbiComp/ISWC '19 Adjunct, 2019 9
Tasks: Activity Recognition, Multimodal Activity Recognition, Time Series, Time Series Analysis

For the Nurse Care Activity Recognition Challenge, an activity recognition algorithm was developed by Team TDU-DSML. A spatial-temporal graph convolutional network (ST-GCN) was applied to process 3D motion capture data included in the challenge dataset. Time-series data was divided into 20-second segments with a 10-second overlap. The recognition model with a tree-structure graph was then created. The prediction result was set to one-minute segments on the basis of a majority decision from each segment output. Our model was evaluated by using leave-one-subject-out cross-validation methods. An average accuracy of 57% for all six subjects was achieved.
Nurse care activity recognition challenge: summary and results
Authors: http://delivery.acm.org/10.1145/3350000/3345577/p746-lago.pdf
Date: 2019-09-09 00:00:00
Proceeding: UbiComp/ISWC '19 Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers 2019 9
Tasks: Activity Recognition, Multimodal Activity Recognition

Although activity recognition has been studied for a long time now, research and applications have focused on physical activity recognition. Even if many application domains require the recognition of more complex activities, research on such activities has attracted less attention. One reason for this gap is the lack of datasets to evaluate and compare different methods. To promote research in such scenarios, we organized the Open Lab Nursing Activity Recognition Challenge focusing on the recognition of complex activities related to the nursing domain. Nursing domain is one of the domains that can benefit enormously from activity recognition but has not been researched due to lack of datasets. The competition used the CARE-COM Nurse Care Activity Dataset, featuring 7 activities performed by 8 subjects in a controlled environment with accelerometer sensors, motion capture and indoor location sensor. In this paper, we summarize the results of the competition.
Can a simple approach identify complex nurse care activity?
Authors: Md. Eusha Kadir, Pritom Saha Akash, Sadia Sharmin, Amin Ahsan Ali, Mohammad Shoyaib
Date: 2019-09-09 00:00:00
Proceeding: UbiComp/ISWC '19 Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers 2019 9
Tasks: Activity Recognition, Multimodal Activity Recognition

For the last two decades, more and more complex methods have been developed to identify human activities using various types of sensors, e.g., data from motion capture, accelerometer, and gyroscopes sensors. To date, most of the researches mainly focus on identifying simple human activities, e.g., walking, eating, and running. However, many of our daily life activities are usually more complex than those. To instigate research in complex activity recognition, the "Nurse Care Activity Recognition Challenge" [1] is initiated where six nurse activities are to be identified based on location, air pressure, motion capture, and accelerometer data. Our team, "IITDU", investigates the use of simple methods for this purpose. We first extract features from the sensor data and use one of the simplest classifiers, namely K-Nearest Neighbors (KNN). Experiment using an ensemble of KNN classifiers demonstrates that it is possible to achieve approximately 87% accuracy on 10-fold cross-validation and 66% accuracy on leave-one-subject-out cross-validation.
Action recognition from depth maps using deep convolutional neural networks
Authors: Pichao Wang, Wanqing Li, Zhimin Gao, Jing Zhang, Chang Tang, Philip Ogunbona
Date: 2015-11-01 00:00:00
Proceeding: IEEE Transactions on Human-Machine Systems 2016 2015 11
Tasks: Action Recognition, Multimodal Activity Recognition, Temporal Action Localization

This paper proposes a new method, i.e., weighted hierarchical depth motion maps (WHDMM) + three-channel deep convolutional neural networks (3ConvNets), for human action recognition from depth maps on small training datasets. Three strategies are developed to leverage the capability of ConvNets in mining discriminative features for recognition. First, different viewpoints are mimicked by rotating the 3-D points of the captured depth maps. This not only synthesizes more data, but also makes the trained ConvNets view-tolerant. Second, WHDMMs at several temporal scales are constructed to encode the spatiotemporal motion patterns of actions into 2-D spatial structures. The 2-D spatial structures are further enhanced for recognition by converting the WHDMMs into pseudocolor images. Finally, the three ConvNets are initialized with the models obtained from ImageNet and fine-tuned independently on the color-coded WHDMMs constructed in three orthogonal planes. The proposed algorithm was evaluated on the MSRAction3D, MSRAction3DExt, UTKinect-Action, and MSRDailyActivity3D datasets using cross-subject protocols. In addition, the method was evaluated on the large dataset constructed from the above datasets. The proposed method achieved 2-9% better results on most of the individual datasets. Furthermore, the proposed method maintained its performance on the large dataset, whereas the performance of existing methods decreased with the increased number of actions.
Beyond Word Attention: Using Segment Attention in Neural Relation Extraction
Authors: Bowen Yu, Zhen-Yu Zhang, Tingwen Liu, Bin Wang, Sujian Li, Quangang Li
Date: 2019-08-10 00:00:00
Proceeding: IJCAI-19 2019 8
Tasks: Relation, Relation Extraction, Sentence

Relation extraction studies the issue of predicting semantic relations between pairs of entities in sentences. Attention mechanisms are often used in this task to alleviate the inner-sentence noise by performing soft selections of words independently. Based on the observation that information pertinent to relations is usually contained within segments (continuous words in a sentence), it is possible to make use of this phenomenon for better extraction. In this paper, we aim to incorporate such segment information into neural relation extractor. Our approach views the attention mechanism as linear-chain conditional random fields over a set of latent variables whose edges encode the desired structure, and regards attention weight as the marginal distribution of each word being selected as a part of the relational expression. Experimental results show that our method can attend to continuous relational expressions without explicit annotations, and achieve the state-of-the-art performance on the large-scale TACRED dataset.
Joint extraction of entities and overlapping relations using position-attentive sequence labeling
Authors: Dai Dai, Xinyan Xiao, Yajuan Lyu, Shan Dou, Qiaoqiao She, Haifeng Wang
Date: 2019-07-17 00:00:00
Proceeding: AAAI-2019 2019 7
Tasks: Joint Entity and Relation Extraction, Position, Relation, Relation Extraction, Sentence, TAG

Joint entity and relation extraction is to detect entity and relation using a single model. In this paper, we present a novel unified joint extraction model which directly tags entity and relation labels according to a query word position p, i.e., detecting entity at p, and identifying entities at other positions that have relationship with the former. To this end, we first design a tagging scheme to generate n tag sequences for an n-word sentence. Then a position-attention mechanism is introduced to produce different sentence representations for every query position to model these n tag sequences. In this way, our method can simultaneously extract all entities and their type, as well as all overlapping relations. Experiment results show that our framework performances significantly better on extracting overlapping relations as well as detecting long-range relation, and thus we achieve state-of-the-art performance on two public datasets.
Jointly Extracting Multiple Triplets with Multilayer Translation Constraints
Authors: Zhen Tan, Xiang Zhao, Wei Wang, Weidong Xiao
Date: 2019-07-17 00:00:00
Proceeding: AAAI-2019 2019 7
Tasks: Knowledge Base Construction, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Relation Classification, Relation Extraction, Sentence, Translation

Triplets extraction is an essential and pivotal step in automatic knowledge base construction, which captures structural information from unstructured text corpus. Conventional extraction models use a pipeline of named entity recognition and relation classification to extract entities and relations, respectively, which ignore the connection between the two tasks. Recently, several neural network-based models were proposed to tackle the problem, and achieved state-of-the-art performance. However, most of them are unable to extract multiple triplets from a single sentence, which are yet commonly seen in real-life scenarios. To close the gap, we propose in this paper a joint neural extraction model for multitriplets, namely, TME, which is capable of adaptively discovering multiple triplets simultaneously in a sentence via ranking with translation mechanism. In experiment, TME exhibits superior performance and achieves an improvement of 37.6% on F1 score over state-of-the-art competitors.
A Survey of FPGA Based Deep Learning Accelerators: Challenges and Opportunities
Authors: Teng Wang, Chao Wang, Xuehai Zhou, Huaping Chen
Date: 2018-12-25 00:00:00
Tasks: CPU, Deep Learning, GPU

With the rapid development of in-depth learning, neural network and deep learning algorithms have been widely used in various fields, e.g., image, video and voice processing. However, the neural network model is getting larger and larger, which is expressed in the calculation of model parameters. Although a wealth of existing efforts on GPU platforms currently used by researchers for improving computing performance, dedicated hardware solutions are essential and emerging to provide advantages over pure software solutions. In this paper, we systematically investigate the neural network accelerator based on FPGA. Specifically, we respectively review the accelerators designed for specific problems, specific algorithms, algorithm features, and general templates. We also compared the design and implementation of the accelerator based on FPGA under different devices and network models and compared it with the versions of CPU and GPU. Finally, we present to discuss the advantages and disadvantages of accelerators on FPGA platforms and to further explore the opportunities for future research.