Reviewerly/exHarmony huggingface dataset:
Reliable Age and Gender Estimation from Face Images: Stating the Confidence of Model Predictions

Automated age and gender estimation became of great importance for many potential applications ranging from forensics to social media. Although previous works reported high increased performances, these solutions tend to mispredict under challenging conditions or when the trained model faces a sample that was underrepresented in the training data. In this work, we propose an age and gender estimation model, as well as a novel reliability measure to quantify the confidence of the model's prediction. Our solution is based on stochastic forward passes through dropout-reduced neural networks that were theoretically proven to approximate Gaussian processes. By utilizing multiple stochastic forward passes, the centrality and dispersion of these predictions are used to derive a confidence statement about the prediction. Experiments were conducted on the Adience benchmark. We showed that the proposed solution reached and exceeded state-of-the-art performance. Further, we demonstrated that the proposed reliability measure correlates with the prediction performance and thus, is highly successful in quantifying the prediction reliability.
A Survey of Image Synthesis Methods for Visual Machine Learning

Abstract Image synthesis designed for machine learning applications provides the means to efficiently generate large quantities of training data while controlling the generation process to provide the best distribution and content variety. With the demands of deep learning applications, synthetic data have the potential of becoming a vital component in the training pipeline. Over the last decade, a wide variety of training data generation methods has been demonstrated. The potential of future development calls to bring these together for comparison and categorization. This survey provides a comprehensive list of the existing image synthesis methods for visual machine learning. These are categorized in the context of image generation, using a taxonomy based on modelling and rendering, while a classification is also made concerning the computer vision applications they are used. We focus on the computer graphics aspects of the methods, to promote future image generation for machine learning. Finally, each method is assessed in terms of quality and reported performance, providing a hint on its expected learning potential. The report serves as a comprehensive reference, targeting both groups of the applications and data development sides. A list of all methods and papers reviewed herein can be found at https://computergraphics.on.liu.se/image_synthesis_methods_for_visual_machine_learning/ .
Path Planning for Mobile Robot Navigation in Unknown Indoor Environments Using Hybrid PSOFS Algorithm

Over the past few years, mobile robots are widely used in various industries because they can navigate in dynamic environments and carry out everyday tasks efficiently. Path planning undoubtedly plays an important role in mobile robot navigation, thus becoming one of the most researched topics in the field of robotics. The metaheuristic algorithms have been extensively studied in recent years to solve the path planning problems the same way as the optimization problems were solved, or in other words, plan the optimum path for the robot to navigate from the starting point to the destination. In this research, a mobile robot is launched to find its path from the starting point to the destination in three different simulated environments using a proposed hybrid metaheuristic algorithm between Particle Swarm Optimization and Fringe Search Algorithm, named PSOFS, and its simultaneous localization and mapping (SLAM) capability. During runtime, the path is optimized by considering the path length. The performance of PSOFS for local path planning is compared against two existing algorithms by evaluating the path smoothness as well as robot safety. The results showed that PSOFS has successfully generated shorter, smoother, and safer paths than the algorithms for mobile robot navigation in unknown indoor environments.
An adaptive image inpainting method based on euler's elastica with adaptive parameters estimation and the discrete gradient method

Euler's Elastica is a common approach developed based on minimizing the elastica energy. It is one of the effective approaches to solve the image inpainting problem. Nevertheless, there are two major issues: the Euler's elastica variational image inpainting model itself is multiparameter, and the performance of methods for solving the model is not high. In the article, we propose an adaptive Euler's elastica image inpainting model by combining with adaptive parameter estimation based on the smoothed structure tensor. To implement the model, a numerical algorithm based on the discrete gradient method is developed. The experiments showed that the proposed image inpainting method outperforms other state-of-the-arts methods in terms of inpainted image quality.
TDFSSD: Top-Down Feature Fusion Single Shot MultiBox Detector

Object detection across different scales is challenging as the variances of object scales. Thus, a novel detection network, Top-Down Feature Fusion Single Shot MultiBox Detector (TDFSSD), is proposed. The proposed network is based on Single Shot MultiBox Detector (SSD) using VGG-16 as backbone with a novel, simple yet efficient feature fusion module, namely, the Top-Down Feature Fusion Module. The proposed module fuses features from higher-level features, containing semantic information, to lower-level features, containing boundary information, iteratively. Extensive experiments have been conducted on PASCAL VOC2007, PASCAL VOC2012, and MS COCO datasets to demonstrate the efficiency of the proposed method. The proposed TDFSSD network is trained end to end and outperforms the state-of-the-art methods across the three datasets. The TDFSSD network achieves 81.7% and 80.1% mAPs on VOC2007 and 2012 respectively, which outperforms the reported best results of both one-stage and two-stage frameworks. In the meantime, it achieves 33.4% mAP on MS COCO test-dev, especially 17.2% average precision (AP) on small objects. Thus all the results show the efficiency of the proposed method on object detection. Code and model are available at: https://github.com/dongfengxijian/TDFSSD.
APNAS: Accuracy-and-Performance-Aware Neural Architecture Search for Neural Hardware Accelerators

Designing resource-efficient deep neural networks (DNNs) is a challenging task due to the enormous diversity of applications as well as their time-consuming design, training, optimization, and evaluation cycles, especially the resource-constrained embedded systems. To address these challenges, we propose a novel DNN design framework called accuracy-and-performance-aware neural architecture search (APNAS), which can generate DNNs efficiently, as it does not require hardware devices or simulators while searching for optimized DNN model configurations that offer both inference accuracy and high execution performance. In addition, to accelerate the process of DNN generation, APNAS is built on a weight sharing and reinforcement learning-based exploration methodology, which is composed of a recurrent neural network controller as its core to generate sample DNN configurations. The reward in reinforcement learning is formulated as a configurable function to consider the sample DNNs’ accuracy and cycle count required to run on a target hardware architecture. To further expedite the DNN generation process, we devise analytical models for cycle count estimation instead of running millions of DNN configurations on real hardware. We demonstrate that these analytical models are highly accurate and provide cycle count estimates identical to those of a cycle-accurate hardware simulator. Experiments that involve quantitatively varying hardware constraints demonstrate that APNAS requires only 0.55 graphics processing unit (GPU) days on a single Nvidia GTX 1080Ti GPU to generate DNNs that offer an average of 53% fewer cycles with negligible accuracy degradation (on average 3%) for image classification compared to state-of-the-art techniques.
A Novel Deep Learning and Polar Transformation Framework for an Adaptive Automatic Modulation Classification

Automatic Modulation Classification (AMC) is an approach to identify an observed signal's most likely modulation scheme without any a priori knowledge of the intercepted signal. In this research, the authors present a new direction for both stages of feature-based (FB) approach. In the feature extraction stage, the authors design a new architecture that 1) removes the bias issue for the estimator of fourth-order cumulants, and 2) extracts polar-transformed information of the received $IQ$ symbols, and finally 3) forms a unique dataset to be used in the labeling stage. Furthermore, the authors contribute to increasing the classification accuracy in low signal-to-noise ratio (SNR) conditions by employing the deep belief network (DBN) platform in addition to the spiking neural network (SNN) platform to overcome execution latency concerns associated with deep learning architectures. For this research, the authors first study each individual FB AMC classifier to derive their respective upper and lower performance bounds and then propose an adaptive framework that is built and developed with these findings. This framework aims to efficiently classify the modulation scheme by intelligently switching between these different FB classifiers to achieve an optimal balance between accuracy and execution latency for any observed channel conditions derived from the main receiver's equalizer. Subsequently, a performance analysis is conducted using the standard RadioML dataset to achieve a realistic evaluation. Numerical results indicate a notably higher classification accuracy, by 16.02% on average, when DBN is employed, whereas SNN requires significantly lower execution latency to label the modulation scheme when compared against two other modified FB classifiers that are built upon convolutional and recurrent neural networks, shown to be reduced by 34.31%.
Deep Learning Enabled Reliable Identity Verification and Spoofing Detection

Identity spoofing is one of the severe threats in Cyber-Physical Systems (CPS). These attacks can cause hazardous issues if they occur in Air Traffic Control (ATC) or other safety-critical systems. For example, a malicious UAV (Unmanned Aerial Vehicle) could easily impersonate a legitimate aircraft's identifier to trespass controlled airspace or broadcast falsified information to disable the airspace operation. In this paper, we propose a joint solution of identity verification and spoofing detection in ATC with assured performances. First, we use an enhanced Deep Neural Network with a zero-bias dense layer with an interpretable mechanism. Second, based on the enhanced Deep Neural Network, we reliably verify airborne targets' identity with the capability of detecting spoofing attacks. Third, we provide a continual learning paradigm to enable the dynamic evolution of the Deep Neural Network. Our approaches are validated using real Automatic dependent surveillance–broadcast (ADS–B) signals and can be generalized to secure other safety-critical CPS.
Automatic Pancreas Segmentation in CT Images With Distance-Based Saliency-Aware DenseASPP Network

Pancreas identification and segmentation is an essential task in the diagnosis and prognosis of pancreas disease. Although deep neural networks have been widely applied in abdominal organ segmentation, it is still challenging for small organs (e.g. pancreas) that present low contrast, highly flexible anatomical structure and relatively small region. In recent years, coarse-to-fine methods have improved pancreas segmentation accuracy by using coarse predictions in the fine stage, but only object location is utilized and rich image context is neglected. In this paper, we propose a novel distance-based saliency-aware model, namely DSD-ASPP-Net, to fully use coarse segmentation to highlight the pancreas feature and boost accuracy in the fine segmentation stage. Specifically, a DenseASPP (Dense Atrous Spatial Pyramid Pooling) model is trained to learn the pancreas location and probability map, which is then transformed into saliency map through geodesic distance-based saliency transformation. In the fine stage, saliency-aware modules that combine saliency map and image context are introduced into DenseASPP to develop the DSD-ASPP-Net. The architecture of DenseASPP brings multi-scale feature representation and achieves larger receptive field in a denser way, which overcome the difficulties brought by variable object sizes and locations. Our method was evaluated on both public NIH pancreas dataset and local hospital dataset, and achieved an average Dice-Sørensen Coefficient (DSC) value of 85.49±4.77% on the NIH dataset, outperforming former coarse-to-fine methods.
Steganography using a 3-player game

Image steganography aims to securely embed secret information into cover images. Until now, adaptive embedding algorithms such as S-UNIWARD or Mi-POD, were among the most secure and most often used methods for image steganography. With the arrival of deep learning and more specifically, Generative Adversarial Networks (GAN), new steganography techniques have appeared. Among them is the 3-player game approach, where three networks compete against each other. In this paper, we propose three different architectures based on the 3-player game. The first architecture is proposed as a rigorous alternative to two recent publications. The second takes into account stego noise power. Finally, our third architecture enriches the second one with a better interaction between embedding and extracting networks. Our method achieves better results compared to existing works Hayes and Danezis (2017), Zhu et al. (2018), and paves the way for future research on this topic.
A Literature Review of Automatic Traceability Links Recovery for Software Change Impact Analysis

In large-scale software development projects, change impact analysis (CIA) plays an important role in controlling software design evolution. Identifying and accessing the effects of software changes using traceability links between various software artifacts is a common practice during the software development cycle. Recently, research in automated traceability-link recovery has received broad attention in the software maintenance community to reduce the manual maintenance cost of trace links by developers. In this study, we conducted a systematic literature review related to automatic traceability link recovery approaches with a focus on CIA. We identified 33 relevant studies and investigated the following aspects of CIA: traceability approaches, CIA sets, degrees of evaluation, trace direction and methods for recovering traceability link between artifacts of different types. Our review indicated that few traceability studies focused on designing and testing impact analysis sets, presumably due to the scarcity of datasets. Based on the findings, we urge further industrial case studies. Finally, we suggest developing traceability tools to support fully automatic traceability approaches, such as machine learning and deep learning.
Spectrum-aware discriminative deep feature learning for multi-spectral face recognition

One primary challenge of face recognition is that the performance is seriously affected by varying illumination. Multi-spectral imaging can capture face images in the visible spectrum and beyond, which is deemed to be an effective technology in response to this challenge. For current multi-spectral imaging-based face recognition methods, how to fully explore the discriminant and correlation features from both the intra-spectrum and inter-spectrum aspects with only a limited number of multi-spectral samples for model training has not been well studied. To address this problem, in this paper, we propose a novel face recognition approach named Spectrum-aware Discriminative Deep Learning (SDDL). To take full advantage of the multi-spectral training samples, we build a discriminative multi-spectral network (DMN) and take face sample pairs as the input of the network. By jointly considering the spectrum and the class label information, SDDL trains the network for projecting samples pairs into a discriminant feature subspace, on which the intrinsic relationship including the intra- and inter-spectrum discrimination and the inter-spectrum correlation among face samples is well discovered. The proposed approach is evaluated on three widely used datasets HK PolyU, CMU, and UWA. Extensive experimental results demonstrate the superiority of SDDL over state-of-the-art competing methods.
Listen to Your Fingers

Inputting a pattern or PIN code on the touch screen is a popular method to prevent unauthorized access to mobile devices. However, these sensitive tokens are highly susceptible to being inferred by various types of side-channel attacks, which can compromise the security of the private data stored in the device. This paper presents a second-factor authentication method, TouchPrint, which relies on the user's hand posture shape traits (dependent on the individual different posture type and unique hand geometry biometrics) when the user inputs PIN or pattern. It is robust against the behavioral variability of inputting a passcode and places no restrictions on input manner (e.g., number of the finger touching the screen, moving speed, or pressure). To capture the spatial characteristic of the user's hand posture shape when input the PIN or pattern, TouchPrint performs active acoustic sensing to scan the user's hand posture when his/her finger remains static at some reference positions on the screen (e.g., turning points for the pattern and the number buttons for the PIN code), and extracts the multipath effect feature from the echo signals reflected by the hand. Then, TouchPrint fuses with the spatial multipath feature-based identification results generated from the multiple reference positions to facilitate a reliable and secure MFA system. We build a prototype on smartphone and then evaluate the performance of TouchPrint comprehensively in a variety of scenarios. The experiment results demonstrate that TouchPrint can effectively defend against the replay attacks and imitate attacks. Moreover, TouchPrint can achieve an authentication accuracy of about 92% with only ten training samples.
Deep understanding of shopper behaviours and interactions using RGB-D vision

Abstract In retail environments, understanding how shoppers move about in a store’s spaces and interact with products is very valuable. While the retail environment has several favourable characteristics that support computer vision, such as reasonable lighting, the large number and diversity of products sold, as well as the potential ambiguity of shoppers’ movements, mean that accurately measuring shopper behaviour is still challenging. Over the past years, machine-learning and feature-based tools for people counting as well as interactions analytic and re-identification were developed with the aim of learning shopper skills based on occlusion-free RGB-D cameras in a top-view configuration. However, after moving into the era of multimedia big data, machine-learning approaches evolved into deep learning approaches, which are a more powerful and efficient way of dealing with the complexities of human behaviour. In this paper, a novel VRAI deep learning application that uses three convolutional neural networks to count the number of people passing or stopping in the camera area, perform top-view re-identification and measure shopper–shelf interactions from a single RGB-D video flow with near real-time performances has been introduced. The framework is evaluated on the following three new datasets that are publicly available: TVHeads for people counting, HaDa for shopper–shelf interactions and TVPR2 for people re-identification. The experimental results show that the proposed methods significantly outperform all competitive state-of-the-art methods (accuracy of 99.5% on people counting, 92.6% on interaction classification and 74.5% on re-id), bringing to different and significative insights for implicit and extensive shopper behaviour analysis for marketing applications.
Monocular vision based obstacle avoidance trajectory planning for Unmanned Aerial Vehicle

In recent years, monocular vision has been considered as an effective Unmanned Aerial Vehicle (UAV) onboard obstacle perception solution with applications to Sense and Avoid (SAA). However, with the limitations of monocular optical measurement, monocular vision based obstacle localization ability is insufficient for collision avoidance. Therefore, an obstacle collision avoidance trajectory planning scheme is proposed in this paper with considerations of monocular optical measurement characteristic. Firstly, two obstacle localization modes are defined in this paper to assist with obstacle collision avoidance, namely relative range based Mode 1 and relative angle based Mode 2. Obstacle localization observability of Mode 2 is analyzed, and coordinate systems for obstacle localization are constructed based on the analysis results. Given coordinate systems, Orthogonal Iteration (OI) is further adopted for obstacle localization. Secondly, due to the lack of global knowledge caused by monocular vision based localization capability, a rolling horizon based safe trajectory planning method is presented. In each time segment, the trajectory is optimized with considerations of: 1) objective functions including minimum trajectory length, elapsed time and energy consumption; 2) constraints including Mode 1 and 2 based obstacle collision avoidance constrains and UAV flight constraints. Finally, simulation results indicate the proposed obstacle collision avoidance trajectory planning scheme enhances UAV safety level and can achieve favorable performance when compared with geometric based obstacle collision avoidance and collision avoidance with global knowledge.
Learning semi-lazy Bayesian network classifier under the c.i.i.d assumption

Bayesian network classifiers (BNCs) are powerful tools in knowledge representation and inference under conditions of uncertainty. In contrast to eager learning, lazy learning seeks to improve the classification accuracy of BNCs by forming a decision theory that is especially tailored for the testing instance, whereas it has received less attention due to the high computational cost at classification time. This study introduces the conditionally independently and identically distributed (c.i.i.d.) assumption to BNCs by assuming that all instances of the same class are conditionally independent of each other and stem from the same probability distribution. Based on this premise, we propose a novel lazy BNC, semi-lazy Bayesian network classifier (SLB), which transforms each unlabeled testing instance to a series of complete instances with discriminative supposed class labels, and then builds class-specific local BNCs for each of them. Our experimental comparison on 25 UCI datasets shows that SLB has modest training time overheads and less classification time overheads. The Friedman and Nemenyi tests show that SLB has significant zero–one loss and bias advantages over some state-of-the-art BNCs, such as selective k-dependence Bayesian classifier, k-nearest neighbor, lazy Bayesian rule and average n-dependence estimators with lazy subsumption resolution.