Reviewerly/exHarmony huggingface dataset:
Hand Gesture recognition using Deep Learning

Sign language is the principal mode of communication for the deaf and dumb community. Conversing with people having such disabilities is a vital challenge faced by normal people. As a result, there is a need for solutions that can considerably reduce the requirement for human interpreters and lead to more easy communication between non-signers and signers. The goal of this study is to create an efficient model that can precisely identify gestures and transform them into a perceivable format. Deep learning models such as CNN (Convolution Neural Network) and YOLOv5 (You Only Look Once) are utilized in this study to effectively predict classes by minimizing the number of parameters of the high dimensional sign picture (as each pixel is considered as a feature). The built models were evaluated using Mean Average Precision. The experimental results indicate that the proposed solution achieves the maximum accuracy, with a Mean Average Precision of 98.8%.
Detection and Recognition of Face Using Deep Learning

Face detection applications using digital photos are critical in the face recognition process. This application is used in biometric recognition systems, search systems, and security systems. Artificial intelligence and machine learning are combined in computer vision. Using computer techniques, it can extract information from images and videos. Many prior studies used various methods and programming languages to create face detection applications. The most crucial aspect of computer vision is object detection. Locating the face is the primary step in computer vision to detect the face part in the input image. For the Java programming language, the Open-Source Computer Vision Library (OpenCV) is a free open-source library for object detection. The Haar cascade classifier is one of the object detection techniques. By counting the number of pictures in a square form on an image, this technique can easily convert an object. The use of face detection in digital photos using the Haar Cascade Classifier and image transformation into grey / grayscale images using the OpenCV library are discussed in this paper. This methodology provides the better investigation accuracy of the outcomes in input photos.
Real-Time Safety Assessment for Dynamic Systems With Limited Memory and Annotations

Real-time safety assessment of dynamic systems has recently received increasing attention. However, the performance of existing advanced approaches is often negatively affected by realistic requirements such as limited annotations and memory. In this case, how to design reasonable query strategies to select valuable instances and exploit the memory space efficiently is extremely meaningful. This paper proposes a novel memory-triggered submodularity-guided active broad learning approach, termed MTSGABL, to deal with such issues simultaneously. Specifically, the broad learning system is introduced as the basic assessment model to update incrementally. A memory-triggered learning mechanism is then proposed based on the drift detection procedure, which controls the update process to exploit the latest sequential information. Furthermore, a submodularity-guided query strategy is introduced to select a small number of valuable samples sequentially, which is beneficial to alleviate the negative effects of the imbalanced data stream. Numerous comparison and ablation experiments with the realistic JiaoLong deep-sea manned submersible data are conducted to validate its effectiveness. Results show that the proposed approach is superior to the existing typical approaches subject to these constraints.
Joint Data Association, Spatiotemporal Bias Compensation and Fusion for Multisensor Multitarget Tracking

Spatiotemporal bias compensation and data association are two essential prerequisites for correct data fusion. Neglect of either in multisensor systems prevents the algorithms from successful application. In this paper, a joint data association, bias compensation and fusion method is presented for multitarget tracking in asynchronous multisensor systems with spatiotemporal biases. The time stamp interval is used to model the state transition, while the difference between the measurement and time stamp interval is regarded as temporal bias and augmented in the state vector as well as spatial bias. The measurements are then formulated as a function of both target states and spatiotemporal biases, where the unknown temporal bias is used to align measurements and target states. Based on the formulation, the joint data association and bias compensation problem is converted into the classical data association and filtering problem in a unified Bayesian framework, without requirement of iterative optimization procedure between data association and bias estimation. The data association is formulated as a 2-D assignment problem and is solved using the generalized Auction algorithm. For each track-measurement pair in association results, the unscented Kalman filter is used to handle the measurement to produce estimates of target states and spatiotemporal bias simultaneously. Furthermore, a bias fusion approach with feedback is presented to fuse the spatiotemporal bias estimates from multiple targets to improve the bias compensation performance and solve the correlation problems therein. Simulation results demonstrate the effectiveness of the proposed method.
Two-Level Scheduling Algorithms for Deep Neural Network Inference in Vehicular Networks

In vehicular networks, task scheduling at the microarchitecture-level and network-level offers tremendous potential to improve the quality of computing services for deep neural network (DNN) inference. However, existing task scheduling works only focus on either one of the two levels, which results in inefficient utilization of computing resources. This paper aims to fill this gap by formulating a two-level scheduling problem for DNN inference tasks in a vehicular network, with an objective of minimizing total weighted sum of response time and energy consumption for all tasks under the following constraints: per task response time, per vehicle energy consumption, per vehicle storage capacity. We first formulate the problem and prove that it is NP-hard. A group transformation based algorithm, called GTA, is proposed. GTA makes scheduling decisions at the network-level using the group transformation based approach, and at the microarchitecture-level using a greedy strategy. In addition, an algorithm, denoted as DRL, is proposed to decrease total weighted sum of response time and energy consumption for all tasks. DRL trains two models with deep reinforcement learning to achieve two-level scheduling. The proposed algorithms are evaluated on a platform consisting of a desktop, Raspberry Pi, Eyeriss, OSM, SUMO, NS-3. Simulation results show that DRL outperforms the state-of-the-art methods for all cases, while the proposed GTA outperforms the state-of-the-art methods for most cases, in terms of total weighted sum of response time and energy consumption. Compared with four baseline algorithms, GTA and DRL reduce the total weighted sum of response time and energy consumption by 41.49% and 62.38%, on average respectively, for different numbers of tasks.
DCAT: Combining Multisemantic Dual-Channel Attention Fusion for Text Classification

Text classification is a fundamental and central position in natural language processing. There are many solutions to the text classification problem, but few use the semantic combination of multiple perspectives to improve the classification performance. The paper proposes a dual-channel attention network model called DCAT, which uses the complementarity between semantics to refine the understanding deficit. Specifically, DCAT first captures the logical semantics of the text through transductive learning and graph structure. Then, at the attention fusion layer (Channel), we use the logical semantics to perform joint semantic training on other semantics to correct the predictions of unlabeled test data incrementally. Experiments show that DCAT can achieve more accurate classification on a wide range of text classification datasets, which is vital for subsequent text mining tasks.
Research on underwater object detection of improved YOLOv7 model based on attention mechanism

Due to the complex content and blurry image of the images obtained in the underwater environment, which affect the detection results, resulting in the problems of false detection of objects and difficulty in feature extraction in conventional object detection models, this paper proposes an improved YOLOv7 model to improve the accuracy and real-time performance of underwater object detection models. Based on the YOLOv7 model, this model integrates the CBAM to enhance the weight of the feature information of the detection object in the spatial dimension and channel dimension, so that the feature information of the model object can be captured more easily and the detection accuracy is improved. The SPPFCSPC module is used to reduce the calculation amount of the model and improve the operation speed of the model while maintaining the receptive field. Through comparative ablation experiments, it is proved that the proposed method has higher detection accuracy in object detection in complex underwater environment than several detection models and YOLOv7 models.
An Auxiliary Decision-Making Method for Autonomous Driving via Monte Carlo Tree Search

The decision-making problem in autonomous driving (AD) is one of the most challenging scenarios, for AD is a real time scenario with a huge space of actions and spaces. Besides there are multiple other agents, it is difficult to predict their actions. In this paper, we propose a fast decision-making auxiliary method based on Monte Carlo Tree Search (MCTS). First, our ego vehicle is controlled by some basic heuristic methods built by humans’ driving experiences, then we use our auxiliary method to improve the ego vehicle's decision-making ability. Besides, we use a pruning technology to reduce time cost. Experiments are carried out on the simulation platform CARLA, the results show that our auxiliary method can improve the decision-making ability of the ego vehicle, and the pruning can make the driving policy stronger.
Chinese Named Entity Recognition Based on BERT and Lexicon Enhancement

Named entity recognition is an important part of information extraction and knowledge graph construction, and is the basic work of natural language processing. Chinese named entity recognition mainly adopts word-based and character-based methods, word-based methods rely on word segmentation and common word segmentation methods have word segmentation errors, which easily cause error propagation, character-based methods avoid this error but do not make full use of lexicon information. The performance of Chinese named entity recognition can be effectively improved by introducing lexicon information into character-based named entity recognition. In this paper, we propose a BERT-IDCNN-CRF model combined with the SoftLexicon method. First, the BERT pre-training language model is used to train the character embedding vector, and the lexicon information is obtained by the SoftLexicon method. Then, the lexicon information is combined with the character vector representation obtained by training. Next, the fused vector representation is input to the IDCNN model for further training. Finally, the recognition results of Chinese named entities are obtained by the CRF model. The experimental results show that the F1 value can reach 95.95%, 70.63% and 95.28% on Resume, Weibo and MSRA datasets, and the training speed is faster than BERT-BiLSTM-CRF.
Pass Strategy of Robocup Robot System Based on Deep Learning Network

In recent years, the research of Multi-Agent Systems (MAS) has become a hot spot in complex systems research. It is widely used in multi-manipulator cooperative equipment, parallel computing, and network resource allocation. The planning of a multi-agent system is more and more critical. This paper takes RoboCup, a classic MAS, as an example. First, it is modeled mathematically through the Markov decision-making process, and a new ball-passing strategy solution based on a deep neural network is proposed. In addition, this paper also tries to use the traditional single-layer neural network to solve the same problem and compares the accuracy difference and model performance of the two types of neural networks. Finally, the possibility of applying a deep learning network in MAS is analyzed. The results show that the deep neural network has high effectiveness in solving the problem of the passing strategy of RoboCup. The ball control rate in the simulation environment is far more than the traditional algorithm, and the model trained by the deep learning network is significantly higher than the traditional neural network in accuracy, efficiency, and other aspects. Applying deep learning networks in other MAS is also likely to have a good effect. In the future, deep learning networks will be an essential planning algorithm.
A novel Swin Transformer based on height class distribution and feature alignment for clothing parsing

The task of clothing parsing is to assign the corresponding clothing category label to each pixel in the clothing image. In this paper, an improved Swin Transformer clothing parsing method fusing height class distribution and feature alignment is proposed to solve the problem that existing clothing parsing algorithms rarely consider position and feature alignment, which leads to unsatisfactory segmentation accuracy. Specifically, we propose a Height Class Distribution Guided Module (HCDGM) to strengthen the guiding role of position information in the training process of clothing praising task. To further improve the parsing accuracy, the feature information of adjacent depth is aligned by Clothing Feature Aligned Module (CFAM). The proposed Swin Transformer based on HCDGM and CFAM can effectively use the height position features of clothing images and correct the information misalignment between different scale features. Experiments results show that our method achieves better results (54.66% of mIoU and 93.93% of PA) on the CFPD dataset compared with other advanced methods.
Experimental Demonstration of Modulation Format Recognition Using Reservoir Computing Based on VCSEL With Gradient-Like Boosting

We present a simple experimental approach based on a photonic time delay reservoir computing (RC) system for modulation format recognition. Here an optically injected vertical cavity surface emitting laser with single feedback is trained with the cross sequence of instantaneous characteristics of modulation signals. Three widely used modulation formats including on–off keying, binary phase shift keying, and binary frequency shift keying, where the optical signal-to-noise ratio varies from 4 dB to 36 dB are considered. Besides, we propose a post-processing method of using multiple gradient-like boosting operations for the time delay RC system, in which the residual value between the target value and the actual value is regarded as the target value for multiple training. Experimental and numerical results demonstrate that using multiple gradient-like boosting operations for post-processing can efficiently identify all three modulation formats with the highest accuracy of 100% after optimizing the control parameters such as the feedback strength, injection strength, and number of virtual nodes of the time delay RC system. The simple experimental approach with the proposed multiple gradient-like boosting operations offers a resource-efficient alternative approach to modulation format recognition of future communication.
Range Extension for Large-Scale Robotic Precision 3-D Measurements in Vibration-Prone Environments

This paper presents a range extension concept for a high-precision robotic inline 3D measurement system. A scanning confocal chromatic sensor (SCCS) is integrated with a magnetically levitated and actuated measurement platform for the acquisition of 3D images with sub-micrometer resolution, enabled by means of active sample-tracking to compensate for relative motion between the SCCS and the sample. Based on an intermediate range extension approach, the lateral scan area is pre-extended to enable a robotic repositioning of the 3D measurement module, ensuring sufficient overlap regions between the individual 3D image frames. Experimental results show that the proposed concept extends the 3D measurement modules’ measurement range by about 80% of the lateral scan area within an acquisition time of about 90 s. Range-extended 3D measurements directly in a vibration-prone environment reveal that 97% of disturbing relative motion between the SCCS and the sample are compensated by the active sample-tracking approach. In this way, lab-like conditions for the SCCS are established directly in an industrial production line and, improving the measurement error by three orders of magnitude down to several tens of nanometers.
Disentangled Representation Learning for RF Fingerprint Extraction Under Unknown Channel Statistics

Deep learning (DL) applied to a device’s radio-frequency fingerprint (RFF) has attracted significant attention in physical-layer authentication due to its extraordinary classification performance. Conventional DL-RFF techniques are trained by adopting maximum likelihood estimation (MLE). Although their discriminability has recently been extended to unknown devices in open-set scenarios, they still tend to overfit the channel statistics embedded in the training dataset. This restricts their practical applications as it is challenging to collect sufficient training data capturing the characteristics of all possible wireless channel environments. To address this challenge, we propose a DL framework of disentangled representation (DR) learning that first learns to factor the signals into a device-relevant component and a device-irrelevant component via adversarial learning. Then, it shuffles these two parts within a dataset for implicit data augmentation, which imposes a strong regularization on RFF extractor learning to avoid the possible overfitting of device-irrelevant channel statistics, without collecting additional data from unknown channels. Experiments validate that the proposed approach, referred to as DR-based RFF, outperforms conventional methods in terms of generalizability to unknown devices under unknown complicated propagation environments, e.g., dispersive multipath fading channels, even though all the training data are collected in a simple environment with dominated direct line-of-sight (LoS) propagation paths.
Lattice-Based Group Signatures With Time-Bound Keys via Redactable Signatures

Group signatures are active cryptographic topics where group members are granted right to sign messages anonymously on behalf of their group. However, in practical applications, such rights are not permanent in most cases and are usually limited to some time periods. This means that the signing right of each group member needs to be associated with time periods such that it can be automatically changed with the latter. Among the known approaches, verifier local revocation (VLR) seems to be the feasible one to implement the above functionality, but it will cause an inefficient verification process when the group size is large. In this paper, we describe a group signature scheme with time-bound keys, based on the hardness of lattice assumption, which implements the limitation of the signing right to any time period by constructing a lattice-based redactable signature scheme. Our scheme still adds VLR mechanism for some members who need to revoke prematurely, but the time-bound keys function ensures such members are only a small fraction that do not incur excessive cost for revocation check. We give implementation for our scheme under 93-bit and 207-bit security respectively to demonstrate the practicability – all costs are independent of the group size and achieve a relatively efficient level.
Eardrum-inspired soft viscoelastic diaphragms for CNN-based speech recognition with audio visualization images

In this study, we present initial efforts for a new speech recognition approach aimed at producing different input images for convolutional neural network (CNN)-based speech recognition. We explored the potential of the tympanic membrane (eardrum)-inspired viscoelastic membrane-type diaphragms to deliver audio visualization images using a cross-recurrence plot (CRP). These images were formed by the two phase-shifted vibration responses of viscoelastic diaphragms. We expect this technique to replace the fast Fourier transform (FFT) spectrum currently used for speech recognition. Herein, we report that the new creation method of color images enabled by combining two phase-shifted vibration responses of viscoelastic diaphragms with CRP shows a lower computation burden and a promising potential alternative way to STFT (conventional spectrogram) when the image resolution (pixel size) is below critical resolution.
CapERA: Captioning Events in Aerial Videos

In this paper, we introduce the CapERA dataset, which upgrades the Event Recognition in Aerial Videos (ERA) dataset to aerial video captioning. The newly proposed dataset aims to advance visual–language-understanding tasks for UAV videos by providing each video with diverse textual descriptions. To build the dataset, 2864 aerial videos are manually annotated with a caption that includes information such as the main event, object, place, action, numbers, and time. More captions are automatically generated from the manual annotation to take into account as much as possible the variation in describing the same video. Furthermore, we propose a captioning model for the CapERA dataset to provide benchmark results for UAV video captioning. The proposed model is based on the encoder–decoder paradigm with two configurations to encode the video. The first configuration encodes the video frames independently by an image encoder. Then, a temporal attention module is added on the top to consider the temporal dynamics between features derived from the video frames. In the second configuration, we directly encode the input video using a video encoder that employs factorized space–time attention to capture the dependencies within and between the frames. For generating captions, a language decoder is utilized to autoregressively produce the captions from the visual tokens. The experimental results under different evaluation criteria show the challenges of generating captions from aerial videos. We expect that the introduction of CapERA will open interesting new research avenues for integrating natural language processing (NLP) with UAV video understandings.