Reviewerly/exHarmony huggingface dataset:
Learning Decision-Making Patterns in the Context of Manned-Unmanned Teaming

Manned-Unmanned Teaming (MUM-T) is an ensemble of manned and unmanned vehicles operating as a team to achieve the same set of goals. Such teaming is highly beneficial, notably in overcoming the limits of direct communication link to the unmanned vehicles, as well as in enhancing capabilities of a manned vehicle by leveraging multiple accompanying unmanned vehicles. However, this can in times overstrain the command and control capacity of the human operator(s) on board of the manned vehicle(s), unless if the unmanned vehicles possess some “understanding” of the human operators’ decision-making behaviors, in which case, they can act like real “team players” to proactively support the manned vehicle, instead of waiting passively for successive commands. In this study, we investigate the possibility of learning decision-making behaviors of a human operator on board of a manned vehicle in charge of commanding multiple accompanying unmanned vehicles. We base our investigation on a rescue mission involving a manned helicopter and several unmanned aerial vehicles to collect training and validation data using software-in-the-loop simulations. By extracting meaningful features and by performing a clustering on the features on the training dataset, we validate and analyze the learned pattern of commanding the unmanned vehicles.
Augmented Reality-Aided Indoor Localization and Navigation for Next-Gen Rescue Services

Augmented reality (AR) technologies experienced a massive upswing in the last decade, with the advent of mobile games like Ingress or Pokémon GO. However, AR's full potential was unlocked only with the introduction of the first holographic devices, projecting images directly in front of the users' eyes. This paper introduces the scenario where the second generation of Microsoft HoloLens, an untethered self-contained holographic device, is used for indoor navigation during fire intervention. The fact that satellite navigation systems are not possible to use indoors highlights the indoor localization capabilities of augmented reality. In our case, we utilized point clouds for navigation. The implemented scenario successfully demonstrates the ability to navigate the person through the building to the final destination. Furthermore, as the technology matures, there is a possibility to see not only the primary point of interest but to give the complete picture of the situation and show, e.g., hydrant positions, main power circuit breakers, and other critical points.
RAS2P: Remote Attestation via Self-Measurement for SGX-based Platforms

Remote Attestation (RA) is a security service by which a Verifier (Vrf) can verify the platform state of a remote Prover (Prv). However, in most existing RA schemes, the Prv might be vulnerable to denial of service (DoS) attacks due to the interactive challenge-response methodology while there is no authentication about the challenge. Worse, many schemes cannot effectively detect mobile malware that can be inactive during the on-demand attestation launched by the Vrf. In this paper, we propose a self-measurement RA for SGX-based platforms, which can effectively mitigate DoS attacks and defend against mobile malware. To this end, a two-way identity authentication is first enforced between the Prv and Vrf with the help of a blockchain system, in which a shared session key is also generated. Secondly, trigger conditions of measurements on the Prv’s side are time points generated by the Prv self instead of Vrf’s requests. The Vrf can retrieve multiple self-measurement results during one execution of the protocol to monitor the Prv’s platform over a period of time continuously, which can detect mobile malware effectively. Our scheme utilizes SGX to provide the runtime protection for sensitive information such as session key, self-measurement code, time points of self-measurements, and self-measurement results, making a higher security guarantee. In addition, the session key, time points of self-measurements, and self-measurement code can be changed or upgraded, making our scheme more flexible and scalable. The simulation implementation and results show that our scheme is feasible and practical.
Designing B-spline-based Highly Efficient Neural Networks for IoT Applications on Edge Platforms

In recent years, with the increasing demand for edge AI, the need for faster AI inference on embedded CPUs is growing. As a result, researchers have focused on various redundancies in neural networks and proposed network acceleration methods. The basic element of most neural networks is a combination of linear transformations and nonlinear activations. We believe this linearity itself is redundant and propose a new method for designing a fast neural network by using nonlinear b-spline functions instead of linear multiplications. We propose an end-to-end design flow for high-speed b-spline neural networks, which shows an actual speedup of 2.4 to 4.1 times compared to conventional neural networks when the network is implemented on Raspberry Pi Zero and Raspberry Pi Pico.
Generating Manipulation Sequences using Reinforcement Learning and Behavior Trees for Peg-In-Hole Task

Reinforcement Learning (RL), a method of learning skills through trial-and-error, has been successfully used in many robotics applications in recent years. We combine manipulation primitives (MPs), behavior trees (BTs), and reinforcement learning to propose an algorithm for peg-in-hole tasks, which speeds up the convergence of the RL model and enhance the adaptability of the dynamic environment. Manipulation primitives are used as actions for RL, which can reduce the gap between control instruction and robotic actions and speed up the convergence of the RL model. Behavior trees are used as robot behavior control, which makes robots can actively adapt to the changes in the environment. In experiments, RL-BT, from the combination of RL and BT, is designed for the peg-in-hole task in the Gazebo simulation environment by using UR5 as the actuator. The experiments are conducted on a simple peg and a complex multi-hole peg by three aspects, which include convergence speed verification experiment, adaptability of dynamic environment experiment, and algorithm robustness experiment. The experiment result proves that our RL-BT can speed up the convergence and adapt to the changes in the environment.
Evolving fuzzy neural network based on null-unineurons for the identification of coronary artery disease

Coronary diseases affect a large part of the world population and have become the target of significant research in the academic field. The creation and use of intelligent models to facilitate the diagnosis of these diseases can allow treatments to be performed promptly to avoid further problems for patients. This paper applies an innovative evolving fuzzy neural network model to solve the problem of coronary heart disease diagnosis and extract valuable insights from the evaluated dataset. The null-unineurons that compose the model’s architecture can extract fuzzy rules, representing linguistic knowledge about the target problem. A dataset that condenses the most famous data sources on this problem to classify coronary heart disease was applied to state-of-the-art models of evolving fuzzy systems. The results obtained by the model applied in this study are similar to the state-of-the-art results. Furthermore, the model provides relevant interpretations about the evolution of the problem evaluation.
Memory Reconstruction Based Dual Encoders for Anomaly Detection

Anomaly detection technology relying on memory reconstruction leverages the difference in reconstruction errors between the normal and abnormal frames to achieve superior detection performance. However, there are still some challenges with this technology. First, the memory has insufficient representation capacity for features. Second, there is a contradiction between feature fusion and reconstruction. As feature fusion copies the abnormal patterns into the reconstructed frames, the abnormal frames are effectively reconstructed, reducing the detection performance. In response to these challenges, we use a memory update threshold to improve the representational power of memory. We also propose a dual-encoder anomaly detection model to restrict anomaly feature propagation. Experiment results demonstrate the effectiveness and robustness of our approach.
Towards the Interpretation of Convolutional Neural Networks for Image Classification Using Fuzzy Sets

Convolutional Neural Networks (CNNs) have demonstrated an outstanding performance on a range of image classification problems in various domains. However, their major drawback is that they are “black box” and opaque classifiers. Taking into consideration the increasing demand for interpretable classification models, this paper introduces a novel meta-feature extraction scheme. This scheme is based on fuzzy sets, and it can be applied on the feature maps of a CNN. Initially, representative image prototypes are selected based on their deep feature map representation. Then, it constructs information granules from the feature maps, describing the content of each image class. It uses fuzzy sets to linguistically characterize the similarity between the deep feature maps of an image and the deep feature maps of the image prototypes. Thus, a classification outcome can be interpreted based on the features characterizing the different image classes involved in a classification problem. The experimental evaluation of the proposed scheme is performed on five publicly available image datasets. The results indicate that the proposed scheme outperforms other state-of-the-art classifiers, while providing an understandable interpretation of the classification result.
Insulator Fault Diagnosis Based on Improved Transfer Learning from UAV Images

Insulator fault diagnosis is a daily but key task for the power transmission system. Long-term exposure to complex natural environment will cause different insulator defects. As a common defects, missing-cap defects of insulators will not only affect the structural strength of power insulators, but also bring a certain effect to the stable power transmission. With the rapid development of machine learning, some machine learning-based defect recognition methods have been proposed for fast and high-precision power inspection. However, the handcrafted features could not effectively express the aerial images against complex inspection environment to affect detection performance of the shallow learning algorithms. And the detection precision of deep learning algorithms will be affected by the unbalanced small-scale defects. Therefore, the fast and high-precision power inspection still faces a certain challenge in the smart grid. To address the above issues, fusion with the deep convolutional neural network (DCNN) and transfer learning, a novel fault diagnosis algorithm of power insulators is proposed to provide a fast and accurate power inspection scheme. To remove complex backgrounds, a fast insulator location algorithm based on the lightweight YOLOV4 model is proposed which is served for the following defect recognition. On the basis, to imitate human vision, a defect recognition algorithm is proposed based on multi-feature fusion. Meanwhile, to ensure the feature expression ability of transfer learning on power insulators, a novel optimization strategy of transfer learning is proposed to improve the recognition precision. Experiments show that the proposed method could acquire a good recognition performance than other recognition models.
Contrast-enhanced Automatic Cognitive Impairment Detection System with Pause-encoder

As the elderly population grows globally, health-care systems face a burden from the rise in Alzheimer’s patients due to an increase in demand for early diagnosis. Therefore, more people have started focusing on developing systems helping doctors diagnose Alzheimer’s, such as cognitive impairment detection systems. This paper presents a contrast-enhanced automatic cognitive impairment screening system combining paused-encoder based on the automatic transcription. We use the pre-trained automatic speech recognition model and adapt it to generate transcripts of the elderly’s speech. The pattern of pauses in speech is a commonly-studied acoustic feature since it can provide additional information besides the semantic information for the model prediction. The back-translation with contrastive learning is used to improve the encoded model further. The model also fine-tunes with the pause-encoded transcriptions to detect the cognitive impairment. Our result shows excellent performance with an accuracy of 81% in detecting Alzheimer’s disease. Also, the accuracy is acceptable on a more challenging task of detecting mild cognitive impairment, the middle stage between healthy and Alzheimer’s. In addition to the outperforming performance, our system is fully automatic and can be used easily.
LBPSC: A Hybrid Prediction Model for Chinese Named Entity Recognition in Water Environment

Recognizing key entities on texts of water environment accurately and rapidly can not only extract important information of water environment, but also improve the water quality. In recent years, Chinese named entity recognition becomes a research focus and many methods based on neural networks have been proven effective on entity recognition. This work proposes an improved hybrid prediction model named LBPSC for Chinese named entity recognition for the water environment data, which combines Lattice structure, Bi-directional long short-term memory (BiLSTM), Positional feature encoding, Sentence self-attention and conditional random field (CRF). LBPSC employs a three-phase end-to-end methodology for Chinese named entity recognition. It first adopts a BiLSTM with lattice structure to extract both character and word features from two directions, thereby avoiding word segmentation errors. It then innovatively combines a sentence self-attention mechanism with positional feature encoding to better handle sentences and add the position information to the trained features after BiLSTM. Then, a CRF layer is adopted to decode features and finally output the predicted tag of the data. Experimental results with real-life dataset demonstrate that LBPSC outperforms other deep learning algorithms in terms of prediction accuracy.
Conflict-Based Search and Improvement Strategies for Solving a New Lexicographic Bi-Objective Multi-Agent Path Finding Problem

Multi-Agent Path Finding (MAPF) is an important problem with a variety of applications. Its aim is to find collision-free paths for agents having separate start and goal positions. This work proposes a new lexicographic bi-objective MAPF considering different task types, where agents are divided into two kinds to perform critical and acritical tasks. This is common in practical intelligent warehousing scenarios where a critical/acritical-task-performing agent (called c-agent and a-agent, respectively) may represent a full-load/no-load one or the one conducting urgent/non-urgent tasks. The primary objective is to minimize the sum-of-costs of c-agents, while the secondary objective is to minimize the sum-of-costs of a-agents. Two MAPF algorithms are modified to fit and solve the concerned problem for the first time. Moreover, four improvement strategies are embedded to the proposed algorithms and proved to be effective in solving MAPF problems with different task types.
A Novel Hierarchical Discourse Model for Scientific Article and It’s Efficient Top-K Resampling-based Text Classification Approach

Scientific articles contain rich knowledge that can significantly assists scientific research, but it is difficult to precisely extract knowledge information due to the complexity of the discourse structure of scientific articles. To provide more accurate scientific research knowledge for researchers in a specific academic domain, it is necessary to study the discourse structure of domain scientific articles and to propose an automatic annotation approach to automatically annotate discourse information from articles. Unfortunately, few works have studied the discourse structure of domain scientific articles and the corresponding automatic discourse annotation. To fill this gap, we take scientific articles of the wastewater-based epidemiology domain as a case to study how to automatically and efficiently annotate discourse information. This paper has three contributions. Firstly, we propose a hierarchical discourse model with two layers to cover all potential discourses in various domain scientific articles. Specifically, the first layer defines four core discourse concepts to describe the main process of a scientific research which can be applied in all scientific articles in various domains. The second layer defines fine-granular domain-specific structure, which can accurately describe the entire research contents of a specific domain. Secondly, based on the proposed model, we build a corpus dataset of 100 annotated scientific articles in the wastewater-based epidemiology domain. Thirdly, based on the model and dataset, we propose a simple yet efficient Top-K resampling-based approach to train a more effective classifier for automatic annotation. Extensive experiments verify the effectiveness and efficiency of our proposed hierarchical discourse model and the Top-K resampling-based classification approach.
Raindrop Removal for In-Vehicle Camera Images with Generative Adversarial Network

Raindrop removal for in-vehicle camera images is useful for surveillance and analysis system such as intelligent driving or case diagnosis. As in rainy days, images taken by in-vehicle cameras such as monitor or drive recorder often suffer from noticeable image degradation, and make it difficult to identify the objects on the road. With the uncertainty of raindrop distribution and the complexity of raindrop status, the attached raindrops of in-vehicle camera images can have different effects. And the large-diameter raindrop will greatly reduce the image quality. However, most of the popular deraining algorithms have been successful in recovering images with small raindrop noise and tiny image distortion, but fails to restore those with raindrops covering large areas. We propose a single image raindrop removal network based on Generative Adversarial Network for in-vehicle camera images. We incorporate two overlapping attention layers into the deraining network, which adopt task-driven visual attention and content perception mechanism. The former obtains features by recursive neural network to guide network more interested in the raindrop regions and the surrounding structures, while the content perception schema extracts the features far from raindrops. Moreover, we collect a new benchmark for In-Vehicle Camera Image Deraining (IVCRID), it chooses in-vehicle camera images from various deraining dataset. The experimental results show that the proposed deraining network outperforms other state-of-the-art raindrop removal methods in image recovering test on the IVCRID dataset and also has the best performance on the experiments of traffic objects detection.
Commonsense-Aware Sarcasm Detection with Heterogeneous Graph Attention Network

Sarcasm is a sophisticated expression, commonly used on social media, e.g., Reddit and Twitter. The presence of sarcasm in social media text flips the polarity of sentiment, thus hindering the performance of works that require true sentiment, e.g., sentiment analysis and opinion mining. However, current works fail to exploit commonsense knowledge in the sarcasm detection task. In this paper, we revisit sarcasm detection from a novel perspective, which models commonsense knowledge as well as context semantics to reason with sarcasm. More specifically, we propose a commonsense-aware model with a heterogeneous graph attention network that leverages commonsense knowledge, enabling it to better understand implied sentiment behind the literal meaning. We conduct experiments on benchmark datasets from Reddit and Internet Argument Corpus. Experimental results show that our proposed approach yields superior performance with commonsense knowledge integrated.
A Secure Approach for Human Computer Interaction Using Human Hand Action

Hand actions classification is an imperative field for acquiring smart functionality in modern electronic devices because hand actions classification offers interactive and innovative methods to communicate and interact. Therefore, we develop a novel architecture based on you only looking at coefficients (YOLACT), a real-time instance segmentation approach, and a temporal relation network (TRN) for hand actions understanding. In addition, our framework consists of a face recognition-based security network (FRB-SN) for user identification. We trained the YOLACT and the TRN models using the segmented version of the 20BN jester dataset composed of hand actions images and ground truths while the FRB-SN is trained using the VGGFace2 dataset. For testing, the YOLACT is used to segment the object from the given image sequence and then passed to the TRN-trained model to predict the corresponding action. Our experimental results showed that the accuracy and frame rate of the proposed framework are competitive.
Dta: An Integrative Approach For Human Action Understanding Based On Region Of Interest

Human action recognition (HAR) is a popular topic in developing a visual analysis system because of its tremendous potential in autonomous visual analysis. However, visual analysis is a sophisticated field in computer vision because an image sequence consists of various features that do not belong to a specific action. Therefore, we present a novel architecture approach for human action recognition and localization. We dubbed it DTA, an abbreviation of the detect, track, and analyze. It is inspired by yolov3, deep-sort, and 3D convolutional neural networks. Our framework is compact in analyzing human action, and the results showed that the proposed method outperforms previous state-of-the-art methods in various aspects. Moreover, the action recognition model is developed, trained, and tested using the ROI version of the KTH dataset. The experimental results showed the accuracy of the proposed model is superior compared to other traditional methods.
Segmentation of Indoor Daily Living Environments into Regions Used for Different Purposes

This article discusses the segmentation of indoor daily living environments, such as rooms in a house, into regions that are habitually used for different purposes by the residents living in the environment. Because these regions usually include purpose-related furnishings, previous studies have focused on the functionalities of these furnishings or their predefined parts to characterize the purposes of the regions, including those furnishings used by the residents. However, the regions used for specific purposes do not necessarily correspond to the furnishings or their predefined parts. Even for a single dining table, its central area typically serves as the region used for placing food, drink, salt shakers etc., whereas the peripheral area typically constitutes a region for dining together. This article attempts to extract the regions used for different purposes as a spatial range from the given environment by segmenting it based on the similarity in the variation of postures observed in each position and the occupancy of its surrounding space. In the experiments using two datasets for indoor daily living environments observed by RGB-D cameras, the obtained regions primarily correspond to our understanding of the regions used for different purposes.