pwc-archive/papers-with-abstracts huggingface dataset:
Black Box Explanation by Learning Image Exemplars in the Latent Feature Space
Authors: Riccardo Guidotti, Anna Monreale, Stan Matwin, Dino Pedreschi
Date: 2020-01-27 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

We present an approach to explain the decisions of black box models for image classification. While using the black box to label images, our explanation method exploits the latent feature space learned through an adversarial autoencoder. The proposed method first generates exemplar images in the latent feature space and learns a decision tree classifier. Then, it selects and decodes exemplars respecting local decision rules. Finally, it visualizes them in a manner that shows to the user how the exemplars can be modified to either stay within their class, or to become counter-factuals by "morphing" into another class. Since we focus on black box decision systems for image classification, the explanation obtained from the exemplars also provides a saliency map highlighting the areas of the image that contribute to its classification, and areas of the image that push it into another class. We present the results of an experimental evaluation on three datasets and two black box models. Besides providing the most useful and interpretable explanations, we show that the proposed method outperforms existing explainers in terms of fidelity, relevance, coherence, and stability.
DFKI Cabin Simulator: A Test Platform for Visual In-Cabin Monitoring Functions
Authors: Hartmut Feld, Bruno Mirbach, Jigyasa Katrolia, Mohamed Selim, Oliver Wasenmüller, Didier Stricker
Date: 2020-01-28 00:00:00
Tasks: 3D Pose Estimation, Intent Detection, Pose Estimation

We present a test platform for visual in-cabin scene analysis and occupant monitoring functions. The test platform is based on a driving simulator developed at the DFKI, consisting of a realistic in-cabin mock-up and a wide-angle projection system for a realistic driving experience. The platform has been equipped with a wide-angle 2D/3D camera system monitoring the entire interior of the vehicle mock-up of the simulator. It is also supplemented with a ground truth reference sensor system that allows to track and record the occupant's body movements synchronously with the 2D and 3D video streams of the camera. Thus, the resulting test platform will serve as a basis to validate numerous in-cabin monitoring functions, which are important for the realization of novel human-vehicle interfaces, advanced driver assistant systems, and automated driving. Among the considered functions are occupant presence detection, size and 3D-pose estimation and driver intention recognition. In addition, our platform will be the basis for the creation of large-scale in-cabin benchmark datasets.
An Overview of Two Age Synthesis and Estimation Techniques
Authors: Milad Taleby Ahvanooey, Qianmu Li
Date: 2020-01-26 00:00:00
Tasks: Age Estimation, Face Verification, Vocal Bursts Valence Prediction

Age estimation is a technique for predicting human ages from digital facial images, which analyzes a person's face image and estimates his/her age based on the year measure. Nowadays, intelligent age estimation and age synthesis have become particularly prevalent research topics in computer vision and face verification systems. Age synthesis is defined to render a facial image aesthetically with rejuvenating and natural aging effects on the person's face. Age estimation is defined to label a facial image automatically with the age group (year range) or the exact age (year) of the person's face. In this case study, we overview the existing models, popular techniques, system performances, and technical challenges related to the facial image-based age synthesis and estimation topics. The main goal of this review is to provide an easy understanding and promising future directions with systematic discussions.
Weighted Average Precision: Adversarial Example Detection in the Visual Perception of Autonomous Vehicles
Authors: Yilan Li, Senem Velipasalar
Date: 2020-01-25 00:00:00
Tasks: Autonomous Driving, Autonomous Vehicles, object-detection, Object Detection

Recent works have shown that neural networks are vulnerable to carefully crafted adversarial examples (AE). By adding small perturbations to input images, AEs are able to make the victim model predicts incorrect outputs. Several research work in adversarial machine learning started to focus on the detection of AEs in autonomous driving. However, the existing studies either use preliminary assumption on outputs of detections or ignore the tracking system in the perception pipeline. In this paper, we firstly propose a novel distance metric for practical autonomous driving object detection outputs. Then, we bridge the gap between the current AE detection research and the real-world autonomous systems by providing a temporal detection algorithm, which takes the impact of tracking system into consideration. We perform evaluation on Berkeley Deep Drive (BDD) and CityScapes datasets to show how our approach outperforms existing single-frame-mAP based AE detections by increasing 17.76% accuracy of performance.
An Empirical Study of Person Re-Identification with Attributes
Authors: Vikram Shree, Wei-Lun Chao, Mark Campbell
Date: 2020-01-25 00:00:00
Tasks: Attribute, Person Re-Identification

Person re-identification aims to identify a person from an image collection, given one image of that person as the query. There is, however, a plethora of real-life scenarios where we may not have a priori library of query images and therefore must rely on information from other modalities. In this paper, an attribute-based approach is proposed where the person of interest (POI) is described by a set of visual attributes, which are used to perform the search. We compare multiple algorithms and analyze how the quality of attributes impacts the performance. While prior work mostly relies on high precision attributes annotated by experts, we conduct a human-subject study and reveal that certain visual attributes could not be consistently described by human observers, making them less reliable in real applications. A key conclusion is that the performance achieved by non-expert attributes, instead of expert-annotated ones, is a more faithful indicator of the status quo of attribute-based approaches for person re-identification.
Music2Dance: DanceNet for Music-driven Dance Generation
Authors: Wenlin Zhuang, Congyi Wang, Siyu Xia, Jinxiang Chai, Yangang Wang
Date: 2020-02-02 00:00:00
Tasks: Diversity, Motion Synthesis, Rhythm

Synthesize human motions from music, i.e., music to dance, is appealing and attracts lots of research interests in recent years. It is challenging due to not only the requirement of realistic and complex human motions for dance, but more importantly, the synthesized motions should be consistent with the style, rhythm and melody of the music. In this paper, we propose a novel autoregressive generative model, DanceNet, to take the style, rhythm and melody of music as the control signals to generate 3D dance motions with high realism and diversity. To boost the performance of our proposed model, we capture several synchronized music-dance pairs by professional dancers, and build a high-quality music-dance pair dataset. Experiments have demonstrated that the proposed method can achieve the state-of-the-art results.
Learning Numerical Observers using Unsupervised Domain Adaptation
Authors: Shenghua He, Weimin Zhou, Hua Li, Mark A. Anastasio
Date: 2020-02-03 00:00:00
Tasks: Domain Adaptation, Image Quality Assessment, Unsupervised Domain Adaptation

Medical imaging systems are commonly assessed by use of objective image quality measures. Supervised deep learning methods have been investigated to implement numerical observers for task-based image quality assessment. However, labeling large amounts of experimental data to train deep neural networks is tedious, expensive, and prone to subjective errors. Computer-simulated image data can potentially be employed to circumvent these issues; however, it is often difficult to computationally model complicated anatomical structures, noise sources, and the response of real world imaging systems. Hence, simulated image data will generally possess physical and statistical differences from the experimental image data they seek to emulate. Within the context of machine learning, these differences between the sets of two images is referred to as domain shift. In this study, we propose and investigate the use of an adversarial domain adaptation method to mitigate the deleterious effects of domain shift between simulated and experimental image data for deep learning-based numerical observers (DL-NOs) that are trained on simulated images but applied to experimental ones. In the proposed method, a DL-NO will initially be trained on computer-simulated image data and subsequently adapted for use with experimental image data, without the need for any labeled experimental images. As a proof of concept, a binary signal detection task is considered. The success of this strategy as a function of the degree of domain shift present between the simulated and experimental image data is investigated.
Improving the Evaluation of Generative Models with Fuzzy Logic
Authors: Julian Niedermeier, Gonçalo Mordido, Christoph Meinel
Date: 2020-02-03 00:00:00
Tasks: Diversity, Image Generation, Sensitivity

Objective and interpretable metrics to evaluate current artificial intelligent systems are of great importance, not only to analyze the current state of such systems but also to objectively measure progress in the future. In this work, we focus on the evaluation of image generation tasks. We propose a novel approach, called Fuzzy Topology Impact (FTI), that determines both the quality and diversity of an image set using topology representations combined with fuzzy logic. When compared to current evaluation methods, FTI shows better and more stable performance on multiple experiments evaluating the sensitivity to noise, mode dropping and mode inventing.
Deriving Emotions and Sentiments from Visual Content: A Disaster Analysis Use Case
Authors: Kashif Ahmad, Syed Zohaib, Nicola Conci, Ala Al-Fuqaha
Date: 2020-02-03 00:00:00
Tasks: Model Selection, Sentiment Analysis

Sentiment analysis aims to extract and express a person's perception, opinions and emotions towards an entity, object, product and a service, enabling businesses to obtain feedback from the consumers. The increasing popularity of the social networks and users' tendency towards sharing their feelings, expressions and opinions in text, visual and audio content has opened new opportunities and challenges in sentiment analysis. While sentiment analysis of text streams has been widely explored in the literature, sentiment analysis of images and videos is relatively new. This article introduces visual sentiment analysis and contrasts it with textual sentiment analysis with emphasis on the opportunities and challenges in this nascent research area. We also propose a deep visual sentiment analyzer for disaster-related images as a use-case, covering different aspects of visual sentiment analysis starting from data collection, annotation, model selection, implementation and evaluations. We believe such rigorous analysis will provide a baseline for future research in the domain.
Towards Deep Machine Reasoning: a Prototype-based Deep Neural Network with Decision Tree Inference
Authors: Plamen Angelov, Eduardo Soares
Date: 2020-02-02 00:00:00

In this paper we introduce the DMR -- a prototype-based method and network architecture for deep learning which is using a decision tree (DT)-based inference and synthetic data to balance the classes. It builds upon the recently introduced xDNN method addressing more complex multi-class problems, specifically when classes are highly imbalanced. DMR moves away from a direct decision based on all classes towards a layered DT of pair-wise class comparisons. In addition, it forces the prototypes to be balanced between classes regardless of possible class imbalances of the training data. It has two novel mechanisms, namely i) using a DT to determine the winning class label, and ii) balancing the classes by synthesizing data around the prototypes determined from the available training data. As a result, we improved significantly the performance of the resulting fully explainable DNN as evidenced by the best reported result on the well know benchmark problem Caltech-101 surpassing our own recently published "world record". Furthermore, we also achieved another "world record" for another very hard benchmark problem, namely Caltech-256 as well as surpassed the results of other approaches on Faces-1999 problem. In summary, we propose a new approach specifically advantageous for imbalanced multi-class problems that achieved two world records on well known hard benchmark problems and the best result on another problem in terms of accuracy. Moreover, DMR offers full explainability, does not require GPUs and can continue to learn from new data by adding new prototypes preserving the previous ones but not requiring full retraining.
Deep Feature Fusion for Mitosis Counting
Authors: Robin Elizabeth Yancey
Date: 2020-02-01 00:00:00
Tasks: Fraud Detection, object-detection, Object Detection, Prognosis, Region Proposal, Segmentation

Each woman living in the United States has about 1 in 8 chance of developing invasive breast cancer. The mitotic cell count is one of the most common tests to assess the aggressiveness or grade of breast cancer. In this prognosis, histopathology images must be examined by a pathologist using high-resolution microscopes to count the cells. Unfortunately, this can be an exhaustive task with poor reproducibility, especially for non-experts. Deep learning networks have recently been adapted to medical applications which are able to automatically localize these regions of interest. However, these region-based networks lack the ability to take advantage of the segmentation features produced by a full image CNN which are often used as a sole method of detection. Therefore, the proposed method leverages Faster RCNN for object detection while fusing segmentation features generated by a UNet with RGB image features to achieve an F-score of 0.508 on the MITOS-ATYPIA 2014 mitosis counting challenge dataset, outperforming state-of-the-art methods.
CONVINCE: Collaborative Cross-Camera Video Analytics at the Edge
Authors: Hannaneh Barahouei Pasandi, Tamer Nadeem
Date: 2020-02-05 00:00:00

Today, video cameras are deployed in dense for monitoring physical places e.g., city, industrial, or agricultural sites. In the current systems, each camera node sends its feed to a cloud server individually. However, this approach suffers from several hurdles including higher computation cost, large bandwidth requirement for analyzing the enormous data, and privacy concerns. In dense deployment, video nodes typically demonstrate a significant spatio-temporal correlation. To overcome these obstacles in current approaches, this paper introduces CONVINCE, a new approach to look at the network cameras as a collective entity that enables collaborative video analytics pipeline among cameras. CONVINCE aims at 1) reducing the computation cost and bandwidth requirements by leveraging spatio-temporal correlations among cameras in eliminating redundant frames intelligently, and ii) improving vision algorithms' accuracy by enabling collaborative knowledge sharing among relevant cameras. Our results demonstrate that CONVINCE achieves an object identification accuracy of $\sim$91\%, by transmitting only about $\sim$25\% of all the recorded frames.
Automatic image-based identification and biomass estimation of invertebrates
Authors: Johanna Ärje, Claus Melvad, Mads Rosenhøj Jeppesen, Sigurd Agerskov Madsen, Jenni Raitoharju, Maria Strandgård Rasmussen, Alexandros Iosifidis, Ville Tirronen, Kristian Meissner, Moncef Gabbouj, Toke Thomas Høye
Date: 2020-02-05 00:00:00
Tasks: Diversity, General Classification, Management

Understanding how biological communities respond to environmental changes is a key challenge in ecology and ecosystem management. The apparent decline of insect populations necessitates more biomonitoring but the time-consuming sorting and identification of taxa pose strong limitations on how many insect samples can be processed. In turn, this affects the scale of efforts to map invertebrate diversity altogether. Given recent advances in computer vision, we propose to replace the standard manual approach of human expert-based sorting and identification with an automatic image-based technology. We describe a robot-enabled image-based identification machine, which can automate the process of invertebrate identification, biomass estimation and sample sorting. We use the imaging device to generate a comprehensive image database of terrestrial arthropod species. We use this database to test the classification accuracy i.e. how well the species identity of a specimen can be predicted from images taken by the machine. We also test sensitivity of the classification accuracy to the camera settings (aperture and exposure time) in order to move forward with the best possible image quality. We use state-of-the-art Resnet-50 and InceptionV3 CNNs for the classification task. The results for the initial dataset are very promising ($\overline{ACC}=0.980$). The system is general and can easily be used for other groups of invertebrates as well. As such, our results pave the way for generating more data on spatial and temporal variation in invertebrate abundance, diversity and biomass.
Level Three Synthetic Fingerprint Generation
Authors: André Brasil Vieira Wyzykowski, Mauricio Pamplona Segundo, Rubisley de Paula Lemes
Date: 2020-02-05 00:00:00

Today's legal restrictions that protect the privacy of biometric data are hampering fingerprint recognition researches. For instance, all high-resolution fingerprint databases ceased to be publicly available. To address this problem, we present a novel hybrid approach to synthesize realistic, high-resolution fingerprints. First, we improved Anguli, a handcrafted fingerprint generator, to obtain dynamic ridge maps with sweat pores and scratches. Then, we trained a CycleGAN to transform these maps into realistic fingerprints. Unlike other CNN-based works, we can generate several images for the same identity. We used our approach to create a synthetic database with 7400 images in an attempt to propel further studies in this field without raising legal issues. We included sweat pore annotations in 740 images to encourage research developments in pore detection. In our experiments, we employed two fingerprint matching approaches to confirm that real and synthetic databases have similar performance. We conducted a human perception analysis where sixty volunteers could hardly differ between real and synthesized fingerprints. Given that we also favorably compare our results with the most advanced works in the literature, our experimentation suggests that our approach is the new state-of-the-art.
Hierarchical Multi-Process Fusion for Visual Place Recognition
Authors: Stephen Hausler, Michael Milford
Date: 2020-01-28 00:00:00
Tasks: Sensor Fusion, Visual Localization, Visual Place Recognition

Combining multiple complementary techniques together has long been regarded as a way to improve performance. In visual localization, multi-sensor fusion, multi-process fusion of a single sensing modality, and even combinations of different localization techniques have been shown to result in improved performance. However, merely fusing together different localization techniques does not account for the varying performance characteristics of different localization techniques. In this paper we present a novel, hierarchical localization system that explicitly benefits from three varying characteristics of localization techniques: the distribution of their localization hypotheses, their appearance- and viewpoint-invariant properties, and the resulting differences in where in an environment each system works well and fails. We show how two techniques deployed hierarchically work better than in parallel fusion, how combining two different techniques works better than two levels of a single technique, even when the single technique has superior individual performance, and develop two and three-tier hierarchical structures that progressively improve localization performance. Finally, we develop a stacked hierarchical framework where localization hypotheses from techniques with complementary characteristics are concatenated at each layer, significantly improving retention of the correct hypothesis through to the final localization stage. Using two challenging datasets, we show the proposed system outperforming state-of-the-art techniques.
Using Fractal Neural Networks to Play SimCity 1 and Conway's Game of Life at Variable Scales
Authors: Sam Earle
Date: 2020-01-29 00:00:00
Tasks: Reinforcement Learning

We introduce gym-city, a Reinforcement Learning environment that uses SimCity 1's game engine to simulate an urban environment, wherein agents might seek to optimize one or a combination of any number of city-wide metrics, on gameboards of various sizes. We focus on population, and analyze our agents' ability to generalize to larger map-sizes than those seen during training. The environment is interactive, allowing a human player to build alongside agents during training and inference, potentially influencing the course of their learning, or manually probing and evaluating their performance. To test our agents' ability to capture distance-agnostic relationships between elements of the gameboard, we design a minigame within the environment which is, by design, unsolvable at large enough scales given strictly local strategies. Given the game engine's extensive use of Cellular Automata, we also train our agents to "play" Conway's Game of Life -- again optimizing for population -- and examine their behaviour at multiple scales. To make our models compatible with variable-scale gameplay, we use Neural Networks with recursive weights and structure -- fractals to be truncated at different depths, dependent upon the size of the gameboard.
Emotion Recognition Using Speaker Cues
Authors: Ismail Shahin
Date: 2020-02-04 00:00:00
Tasks: Emotion Recognition, Quantization

This research aims at identifying the unknown emotion using speaker cues. In this study, we identify the unknown emotion using a two-stage framework. The first stage focuses on identifying the speaker who uttered the unknown emotion, while the next stage focuses on identifying the unknown emotion uttered by the recognized speaker in the prior stage. This proposed framework has been evaluated on an Arabic Emirati-accented speech database uttered by fifteen speakers per gender. Mel-Frequency Cepstral Coefficients (MFCCs) have been used as the extracted features and Hidden Markov Model (HMM) has been utilized as the classifier in this work. Our findings demonstrate that emotion recognition accuracy based on the two-stage framework is greater than that based on the one-stage approach and the state-of-the-art classifiers and models such as Gaussian Mixture Model (GMM), Support Vector Machine (SVM), and Vector Quantization (VQ). The average emotion recognition accuracy based on the two-stage approach is 67.5%, while the accuracy reaches to 61.4%, 63.3%, 64.5%, and 61.5%, based on the one-stage approach, GMM, SVM, and VQ, respectively. The achieved results based on the two-stage framework are very close to those attained in subjective assessment by human listeners.