Reviewerly/exHarmony huggingface dataset:
Understanding How CNNs Recognize Facial Expressions: A Case Study with LIME and CEM

Recognizing facial expressions has been a persistent goal in the scientific community. Since the rise of artificial intelligence, convolutional neural networks (CNN) have become popular to recognize facial expressions, as images can be directly used as input. Current CNN models can achieve high recognition rates, but they give no clue about their reasoning process. Explainable artificial intelligence (XAI) has been developed as a means to help to interpret the results obtained by machine learning models. When dealing with images, one of the most-used XAI techniques is LIME. LIME highlights the areas of the image that contribute to a classification. As an alternative to LIME, the CEM method appeared, providing explanations in a way that is natural for human classification: besides highlighting what is sufficient to justify a classification, it also identifies what should be absent to maintain it and to distinguish it from another classification. This study presents the results of comparing LIME and CEM applied over complex images such as facial expression images. While CEM could be used to explain the results on images described with a reduced number of features, LIME would be the method of choice when dealing with images described with a huge number of features.
An Aeromagnetic Compensation Algorithm Based on Radial Basis Function Artificial Neural Network

Aeromagnetic exploration is a magnetic exploration method that detects changes of the earth’s magnetic field by loading a magnetometer on an aircraft. With the miniaturization of magnetometers and the development of unmanned aerial vehicles (UAV) technology, UAV aeromagnetic surveying plays an increasingly important role in mineral exploration and other fields due to its advantages of low cost and safety. However, in the process of aeromagnetic measurement data, due to the ferromagnetic material of the aircraft itself and the change of flight direction and attitude, magnetic field interference will occur and affect the measurement of the geomagnetic field by the magnetometer. The work of aeromagnetic compensation is to compensate for this part of the magnetic interference and improve the magnetic measurement accuracy of the magnetometer. This paper focused on the problems of UAV aeromagnetic survey data processing and improved the accuracy of UAV based aeromagnetic data measurement. Based on the Tolles–Lawson model, a numerical simulation experiment of magnetic interference of UAV-based aeromagnetic data was carried out, and a radial basis function (RBF) artificial neural network (ANN) algorithm was proposed for the first time to compensate the aeromagnetic data. Compared with classical backpropagation (BP) ANN, the test results of the synthetic data and real measured magnetic data showed that the RBF-ANN has higher compensation accuracy and stronger generalization ability.
Generating Digital Twins for Path-Planning of Autonomous Robots and Drones Using Constrained Homotopic Shrinking for 2D and 3D Environment Modeling

A digital twin describes the virtual representation of a real process. This twin is constantly updated with real data and can thus control and adapt the real model. Designing suitable digital twins for path planning of autonomous robots or drones is often challenging due to the large number of different dynamic environments and multi-task and agent systems. However, common path algorithms are often limited to two tasks and to finding shortest paths. In real applications, not only a short path but also the width of the passage with a path as centered as possible are crucial, since robotic systems are not ideal and require recalibration frequently. In this work, so-called homotopic shrinking is used to generate the digital twin, which can be used to extract all possible path proposals including their passage widths for 2D and 3D environments and multiple tasks and robots. The erosion of the environment is controlled by constraints such that the task stations, the robot or drone positions, and the topology of the environment are considered. Such a deterministic path algorithm can flexibly respond to changing environmental conditions and consider multiple tasks simultaneously for path generation. A distinctive feature of these paths is the central orientation to the non-passable areas, which can have significant benefits for worker and patient safety. The method is tested on 2D and 3D maps with different tasks, obstacles, and multiple robots. For example, the robust generation of the digital twin for a maze and also the dynamic adaptation in case of sudden changes in the environment is covered. This variety of use cases and the comparison with alternative methods result in significant advantages, such as high robustness, consideration of multiple targets, and high safety distances to obstacles and areas that cannot be traversed. Finally, it was shown that the environment for the digital twin can be reduced to reasonable paths by constrained shrinking, both for real 2D maps and for complex virtual 2D and 3D maps.
Universal Linear Optics Revisited: New Perspectives for Neuromorphic Computing With Silicon Photonics

Reprogrammable optical meshes comprise a subject of heightened interest for the execution of linear transformations, having a significant impact in numerous applications that extend from the implementation of optical switches up to neuromorphic computing. Herein, we review the state-of-the-art approaches for the realization of unitary transformations and universal linear operators in the photonic domain and present our recent work in the field, that allows for fidelity restorable and low-loss optical circuitry with single-step programmability. These advantages unlock a new framework for matrix-vector multiplications required by neuromorphic silicon photonic circuits, supporting: i) high-speed and high-accuracy neural network (NN) inference, ii) high-speed tiled matrix multiplication, iii) NN training and iv) programmable photonic NNs. This new potential is initially validated through recent experimental results using SiGe EAM technology and static weights and, subsequently, utilized for demonstrating experimentally the first Deep NN (DNN) where optical tiled matrix multiplication up to 50 GHz is realized, allowing optics to execute DNNs with large number of trainable parameters over a limited photonic hardware. Finally, the new performance framework is benchmarked against state-of-the-art NN processors and photonic NN roadmap projections, highlighting its perspectives to turn the energy and area efficiency promise of neuromorphic silicon photonics into a tangible reality.
A Henon-like chaotic map and its application in image encryption combined with compressed sensing

Abstract A new method of color image encryption is proposed in this paper, with the combination of a two-dimensional (2D) Henon-like chaotic map and compressed sensing for better performance. A sinusoidal function is introduced into the 2D Henon map for increasing randomness. Image encryption algorithm includes three procedures: compression of a color image through the Bernoulli observation matrix, encryption of the compressed image by 2D-Henon map with sinusoidal factor, and reconstruction of color image based on compressed sensing algorithm of orthogonal matching pursuit (OMP). Numerical simulation shows that the proposed encryption algorithm has some advantages including being hardware-friendly, sufficient keyspace, and higher security.
Non-Autoregressive End-to-End Neural Modeling for Automatic Pronunciation Error Detection

A crucial element of computer-assisted pronunciation training systems (CAPT) is the mispronunciation detection and diagnostic (MDD) technique. The provided transcriptions can act as a teacher when evaluating the pronunciation quality of finite speech. The preceding texts have been entirely employed by conventional approaches, such as forced alignment and extended recognition networks, for model development or for enhancing system performance. The incorporation of earlier texts into model training has recently been attempted using end-to-end (E2E)-based approaches, and preliminary results indicate efficacy. Attention-based end-to-end models have shown lower speech recognition performance because multi-pass left-to-right forward computation constrains their practical applicability in beam search. In addition, end-to-end neural approaches are typically data-hungry, and a lack of non-native training data will frequently impair their effectiveness in MDD. To solve this problem, we provide a unique MDD technique that uses non-autoregressive (NAR) end-to-end neural models to greatly reduce estimation time while maintaining accuracy levels similar to traditional E2E neural models. In contrast, NAR models can generate parallel token sequences by accepting parallel inputs instead of left-to-right forward computation. To further enhance the effectiveness of MDD, we develop and construct a pronunciation model superimposed on our approach’s NAR end-to-end models. To test the effectiveness of our strategy against some of the best end-to-end models, we use publicly accessible L2-ARCTIC and SpeechOcean English datasets for training and testing purposes where the proposed model shows the best results than other existing models.
How Engineers’ Imaginaries of Healthcare Shape Design and User Engagement: A Case Study of a Robotics Initiative for Geriatric Healthcare AI Applications

In the development of robotics and Artificial Intelligence (AI) for healthcare, human-centered approaches seek to meet the requirements of healthcare practice and address social and ethical aspects proactively. In this work, an important but neglected aspect of human-computer interaction (HCI) is how engineers understand and envision the healthcare context. Drawing on insights from STS on engineers’ imaginaries and their role in shaping research and development of new technologies, we propose engineers’ imaginaries of healthcare as a point of analysis and intervention for ethical and social aspects of AI and robotics for healthcare. To illustrate the utility of this lens, we use it to on report a case study of an engineering project that develops robotic and AI applications for healthcare. We followed and sought to advance an embedded ethics and social science approach, where ethicists and social scientists accompanied this engineering project using direct interdisciplinary collaboration, observations, and in-depth qualitative interviews with the project’s engineers (n = 18). We analyze how the engineers imagine healthcare as an environment for robots, healthcare workers as potential users, and healthcare practices, and how these imaginaries connect to the design narratives that guide their work. Our findings provide pertinent input for HCI, STS, and engineering ethics related to healthcare AI and robotics, as they speak to prevalent narratives of “assistance” systems, aspects of how human healthcare practices are reframed and valued in the face of new technologies, questions of division of labor between machines and healthcare practitioners, and the implications of ‘acceptance’ as a frame for user-centered design.
Hessian scatter regularized twin support vector machine for semi-supervised classification

Currently, semi-supervised twin support vector machine based on Laplacian regularization (LapTSVM) have received extensive attention and research in many fields of machine learning. Unfortunately, Laplacian regularization has a constant null space, so the solution is often a constant function and cannot well maintain the local topology of the samples. Aiming the above urgent problems, this paper, we first construct a Hessian scatter regularization (HSR) term. HSR has two major advantages: (1) HSR prefers linear variation in function values along of the geodesic distance and maintains the local manifold structure of the samples well. (2) HSR tries to find the projection from the original space to the feature space to maximize the inter-class scatter and minimize the intra-class scatter of the samples; the scatter is regarded as the discriminative information (structural information) of samples. Secondly, by introducing HSR, we propose a Hessian scatter regularized twin support vector machine (HSR-TSVM). Compared with LapTSVM, HSR-TSVM uses the global and local structure information of the sample to overcome the shortcomings of insufficient extrapolation caused by Laplacian regularization, while retaining almost all the advantages of the classic LapTSVM. Furthermore, to improve the computational efficiency of HSR-TSVM, the least-squares version of HSR-TSVM, namely HSR-LSTSVM, is proposed, and the conjugate gradient method is used to solve it. Experimental results on four synthetic datasets, ten UCI datasets, and four image datasets show that the proposed methods are competitive with semi-supervised learning methods based on Laplacian regularization.
A high speed pseudo-random bit generator driven by 2D-discrete hyperchaos

Pseudo-Random Number Generators (PRNGs) yielding numbers with high rates and good randomness quality are crucial for security as networks expand in an ever-connected way. In this work, firstly, we construct a new 2D discrete hyper-chaotic map with linear cross-coupled topological structure combined with the Tent and Logistic map. The proposed map with the aforementioned structure enables it to outperform other enhanced chaotic maps developed recently. Secondly, an efficient PRNG based on the proposed one is implemented on the field-programmable gate array (FPGA) Xilinx xc7k325tffg900-2. Compared with those typical PRNGs, the sequences generated by ours own a high level of randomness and passed the well-known TestU01, Dieharder, and the National Institute of Standards and Technology (NIST) SP800-22 test suite successfully without post-processing. Experimental results show that the proposed PRNG occupies merely 1.4 percent of the resources available on the targeted FPGA despite it yielding numbers with a large bit depth. In addition, the timing report shows the system can operate effectively at a clock of 158 MHz with a maximum throughput of 9.26 Gbps which outperforms the state-of-the-art.
Jellyfish: Timely Inference Serving for Dynamic Edge Networks

While high accuracy is of paramount importance for deep learning (DL) inference, serving inference requests on time is equally critical but has not been carefully studied especially when the request has to be served over a dynamic wireless network at the edge. In this paper, we propose Jellyfish—a novel edge DL inference serving system that achieves soft guarantees on end-to-end inference latency often specified as a service-level objective (SLO). To handle the network variability, Jellyfish exploits both data and deep neural network (DNN) adaptation to conduct tradeoffs between accuracy and latency. Jellyfish features a new design that enables collective adaptation policies where the decisions for data and DNN adaptations are aligned and coordinated among multiple users with varying network conditions. We propose efficient algorithms to dynamically adapt DNNs and map users, so that we fulfill latency SLOs while maximizing the overall inference accuracy. Our experiments based on a prototype implementation and real-world WiFi and LTE network traces show that Jellyfish can meet latency SLOs at around the 99th percentile while maintaining high accuracy.
Demand Layering for Real-Time DNN Inference with Minimized Memory Usage

When executing a deep neural network (DNN), its model parameters are loaded into GPU memory before execution, incurring a significant GPU memory burden. There are studies that reduce GPU memory usage by exploiting CPU memory as a swap device. However, this approach is not applicable in most embedded systems with integrated GPUs where CPU and GPU share a common memory. In this regard, we present Demand Layering, which employs a fast solid-state drive (SSD) as a co-running partner of a GPU and exploits the layer-by-layer execution of DNNs. In our approach, a DNN is loaded and executed in a layer-by-layer manner, minimizing the memory usage to the order of a single layer. Also, we developed a pipeline architecture that hides most additional delays caused by the interleaved parameter loadings alongside layer executions. Our implementation shows a 96.5% memory reduction with just 14.8% delay overhead on average for representative DNNs. Furthermore, by exploiting the memory-delay tradeoff, near-zero delay overhead (under 1 ms) can be achieved with a slightly increased memory usage (still an 88.4% reduction), showing the great potential of Demand Layering.
Two-dimensional motion estimation using phase-based image processing with Riesz transform

As a kind of photogrammetry technique, phase-based motion estimation (PME) has proven to be a powerful image/video processing algorithm in the extraction of structural dynamics, due to its full-filed resolution and no requirement of surface preparation, compared to conventional structural testing. Although images and videos employed in PME are two-dimensional arrays, PME only extracts phase information in a single direction at a time. Therefore, extracting two-directional motions at the same time is needed to avoid two times of PME implementation. To address this issue, Riesz-transform is integrated to the phase-based motion estimation algorithm, and Riesz transform PME (RPME) is proposed in this work. The concept of plane phase is extended to spatial phase using Riesz transform to process images, because Riesz transform is able to extract phase information in two directions simultaneously. Then quaternion is utilized to derive the theoretical relationship between spatial phase and structure motions. In order to obtain the mono-component image sequence required in Riesz transform, an ideal narrow bandpass filter is constructed based on the set of Butterworth low-pass filters. Due to the capability of recognizing two-directional motions simultaneously using the proposed RPME, a much-improved efficiency of structural dynamics identification is observed. In addition, Riesz transform extracts instantaneous phase, thus the motion recognition accuracy is improved as well. To validate the benefit of RPME, both numerical simulation and laboratory-scale experiments are conducted, and motions in x and y directions are extracted simultaneously, with a drastically increased accuracy in both the simulation and experimental validations.
Leveraging image complexity in macro-level neural network design for medical image segmentation

Recent progress in encoder-decoder neural network architecture design has led to significant performance improvements in a wide range of medical image segmentation tasks. However, state-of-the-art networks for a given task may be too computationally demanding to run on affordable hardware, and thus users often resort to practical workarounds by modifying various macro-level design aspects. Two common examples are downsampling of the input images and reducing the network depth or size to meet computer memory constraints. In this paper, we investigate the effects of these changes on segmentation performance and show that image complexity can be used as a guideline in choosing what is best for a given dataset. We consider four statistical measures to quantify image complexity and evaluate their suitability on ten different public datasets. For the purpose of our illustrative experiments, we use DeepLabV3+ (deep large-size), M2U-Net (deep lightweight), U-Net (shallow large-size), and U-Net Lite (shallow lightweight). Our results suggest that median frequency is the best complexity measure when deciding on an acceptable input downsampling factor and using a deep versus shallow, large-size versus lightweight network. For high-complexity datasets, a lightweight network running on the original images may yield better segmentation results than a large-size network running on downsampled images, whereas the opposite may be the case for low-complexity images.
Path Planning of Cleaning Robot with Reinforcement Learning

Recently, as the demand for cleaning robots has steadily increased, therefore household electricity consumption is also increasing. To solve this electricity consumption issue, the problem of efficient path planning for cleaning robot has become important and many studies have been conducted. However, most of them are about moving along a simple path segment, not about the whole path to clean all places. As the emerging deep learning technique, reinforcement learning (RL) has been adopted for cleaning robot. However, the models for RL operate only in a specific cleaning environment, not the various cleaning environment. The problem is that the models have to retrain whenever the cleaning environment changes. To solve this problem, the proximal policy optimization (PPO) algorithm is combined with an efficient path planning that operates in various cleaning environments, using transfer learning (TL), detection nearest cleaned tile, reward shaping, and making elite set methods. The proposed method is validated with an ablation study and comparison with conventional methods such as random and zigzag. The experimental results demonstrate that the proposed method achieves improved training performance and increased convergence speed over the original PPO. And it also demonstrates that this proposed method is better performance than conventional methods (random, zigzag).
Don’t lose samples to estimation

In a typical predictive modeling task, we are asked to produce a final predictive model to employ operationally for predictions, as well as an estimate of its out-of-sample predictive performance. Typically, analysts hold out a portion of the available data, called a Test set, to estimate the model predictive performance on unseen (out-of-sample) records, thus "losing these samples to estimation." However, this practice is unacceptable when the total sample size is low. To avoid losing data to estimation, we need a shift in our perspective: we do not estimate the performance of a specific model instance; we estimate the performance of the pipeline that produces the model. This pipeline is applied on all available samples to produce the final model; no samples are lost to estimation. An estimate of its performance is provided by training the same pipeline on subsets of the samples. When multiple pipelines are tried, additional considerations that correct for the "winner's curse" need to be in place.
A Multimodal Fusion Approach for Human Activity Recognition

The problem of human activity recognition (HAR) has been increasingly attracting the efforts of the research community, having several applications. It consists of recognizing human motion and/or behavior within a given image or a video sequence, using as input raw sensor measurements. In this paper, a multimodal approach addressing the task of video-based HAR is proposed. It is based on 3D visual data that are collected using an RGB + depth camera, resulting to both raw video and 3D skeletal sequences. These data are transformed into six different 2D image representations; four of them are in the spectral domain, another is a pseudo-colored image. The aforementioned representations are based on skeletal data. The last representation is a "dynamic" image which is actually an artificially created image that summarizes RGB data of the whole video sequence, in a visually comprehensible way. In order to classify a given activity video, first, all the aforementioned 2D images are extracted and then six trained convolutional neural networks are used so as to extract visual features. The latter are fused so as to form a single feature vector and are fed into a support vector machine for classification into human activities. For evaluation purposes, a challenging motion activity recognition dataset is used, while single-view, cross-view and cross-subject experiments are performed. Moreover, the proposed approach is compared to three other state-of-the-art methods, demonstrating superior performance in most experiments.