Reviewerly/exHarmony huggingface dataset:
A Modified LSTM Model for Chinese Sign Language Recognition Using Leap Motion

At present, there are about 70 million deaf people using sign language in the world, but for most normal people, it is difficult to understand the meaning of the sign language expression. Therefore, it is of great importance to explore the ways of recognising the sign language. In this paper, we propose a dynamic sign language recognition method based on the modified long short-term memory (LSTM) model. Firstly, we use Leap Motion to collect the features of Chinese Sign Language (CSL). LSTM has a good effect in processing time series data, but the parameters of its hidden layer are shared, making it important information lost when dealing with long time series. The attention mechanism can give different attention weights to different features according to the correlation between the input data and output data, so as to enhance the model’s attention to key information. Therefore, we combine LSTM with attention mechanism for dynamic sign language recognition. Experimental results show that the recognition accuracy of the modified LSTM model is 99.55%, which is higher than that of LSTM model. Finally, we developed a sign language human-computer interaction system, which verifies the real-time performance and effectiveness of the method proposed in this paper.
HTNet: Anchor-free Temporal Action Localization with Hierarchical Transformers

Temporal action localization (TAL) is a task of identifying a set of actions in a video, which involves localizing the start and end frames and classifying each action instance. Existing methods have addressed this task by using predefined anchor windows or heuristic bottom-up boundary-matching strategies, which are major bottlenecks in inference time. Additionally, the main challenge is the inability to capture long-range actions due to a lack of global contextual information. In this paper, we present a novel anchor-free framework, referred to as HTNet, which predicts a set of $\langle$start time, end time, class$\rangle$ triplets from a video based on a Transformer architecture. After the prediction of coarse boundaries, we refine it through a background feature sampling (BFS) module and hierarchical Transformers, which enables our model to aggregate global contextual information and effectively exploit the inherent semantic relationships in a video. We demonstrate how our method localizes accurate action instances and achieves state-of-the-art performance on two TAL benchmark datasets: THUMOS14 and ActivityNet 1.3.
UniMoCo: Unsupervised, Semi-Supervised and Fully-Supervised Visual Representation Learning

Momentum Contrast (MoCo) achieves great success for unsupervised visual representation learning. However, there are a lot of supervised and semi-supervised datasets, which are already labeled. To fully utilize the label annotations, we propose Unified Momentum Contrast (UniMoCo), which extends MoCo to support arbitrary ratios of labeled data and unlabeled data training. Compared with MoCo, UniMoCo has two modifications as follows: (1) Different from a single positive pair in MoCo, we maintain multiple positive pairs on-the-fly by comparing the query label to a label queue. (2) We propose a Unified Contrastive (UniCon) loss to support an arbitrary number of positives and negatives in a unified pair-wise optimization perspective. Our UniCon is more reasonable and powerful than the supervised contrastive loss in theory and practice. In our experiments, we pre-train multiple UniMoCo models with different ratios of ImageNet labels and evaluate the performance on various downstream tasks. Experiment results show that UniMoCo generalizes well for unsupervised, semi-supervised and fully-supervised visual representation learning. Besides, we surprisingly find that UniMoCo performs best with 60% ImageNet labels for COCO and VOC transfer learning. The code is available: https://github.com/dddzg/unimoco.
Multimodal Sentiment Analysis Based on Nonverbal Representation Optimization Network and Contrastive Interaction Learning

Multimodal sentiment analysis is an active subfield of natural language processing. It aims to extract and integrate semantic information gathered from multiple modalities to identify the sentiments expressed by users. Indeed, the complementary and heterogeneous information between modalities influences the prediction results. Recent research proposals employ a single neural network to obtain mutually independent representations of all modalities. However, a problem that may limit previous work from reaching a higher level is that this does not consider the modal heterogeneity problem between different modalities. This, in turn, may lead to the presence of additional noise in the representations before modal fusion. For this reason, we propose a new framework, MICS, which adopts a suitable strategy for each modality and provides a better representation for fusion. Also, we design a multimodal comparative learning interaction module for the fusion phase, which plays a crucial role in the information interaction between modalities. Our extensive experiments on two publicly available and popular benchmarks, MOSI and MOSEI, show that MICS can better focus on the characteristics of different modal data and has significant advantages over previous complex baselines.
RFID and NFC authentication protocol for securing a payment transaction

RFID (Radio Frequency Identification) is a technology that permits to identify objects with a radio frequency field for few meters distance. It is applied in several fields such as traceability, logistics, military, security, healthcare, library, etc. Its successor NFC (Near Field Communication)), on the contrary, is used to transmit messages at a short distance (less than 10 Cent.). Recently, NFC is frequently used for payment transaction using ATM and smartphone both equipped of this technology. Unfortunately, the information transferred with this technology during a payment transaction can be an objectif for threads to steale personal and privacy data that concern a user. This paper introduces a new method for payment transaction that combines RFID and NFC. This process which uses an authentication protocol is proposed in order to (1) secure the payment transaction using an Automated Teller Machine (ATM) and a smartphone both equipped with these two technologies, (2) minimize the waiting chain in front of the ATM for agents using NFC by minimizing the transaction time. To evaluate the performance of our proposal, we tested its security against few attacks. A security and performance comparison of our technique with few recent and important methods is performed and shows the effectiveness of the solution.
Reinforcement learning based web crawler detection for diversity and dynamics

Crawler detection is always an important research topic in network security. With the development of web technology, crawlers are constantly updating and changing, and their types are becoming diverse. The diversity and dynamics of crawlers pose significant challenges for feature applicability and model robustness. Existing crawler detection methods can only detect a limited number of crawlers by predefined rules and can not cover all types of crawlers; worse, they can be completely invalidated by the emergence of new types of crawlers. In this paper, we propose a reinforcement learning based web crawler detection method for diversity and dynamics (WC3D), which is composed of a feature selector and a session classifier. The feature selector selects the appropriate feature set for different types of crawlers with deep deterministic policy gradient. The session classifier makes crawler detection and provides rewards to the feature selector. The two modules are trained jointly to optimize the feature selection and session classification processes. Extensive experiments demonstrate the existence of crawler diversity and that the proposed method is still highly robust against the new type of crawlers and achieves state-of-the-art performance even without considering the dynamics of the crawlers.
DFAF3D:A dual-feature-aware anchor-free single-stage 3D detector for point clouds

Currently, anchor-free single-stage 3D object detection methods based on point clouds have attracted extensive attention due to their high efficiency. It is crucial to enhance the ability of the center features to represent the object for such methods. In this paper, we propose a dual spatial-context feature extraction (SCFE) module to extract both spatial and context features of point clouds in parallel for 3D object detection, in which, we design a deformable offset self-attention (DOSA) structure in the context feature extraction branch to learn the relative structure information of point clouds. Correspondingly, we design a coordinate attentional feature fusion (CAFF) module, which adaptively fuses spatial and context features of different resolutions while preserving coordinate information thus making the features of center point more representative. Experiments on KITTI demonstrate that the proposed method outperforms all previous anchor-free methods in general and has comparable performance to anchor-based methods in comprehensive performance and it achieves good trade-offs in terms of accuracy, speed and parameter size.
Chaotic systems with variable indexs for image encryption application

A new chaotic system is obtained by changing the number of unknown parameters. The dynamical behavior of the chaotic system is investigated by the exponential change of the single unknown parameter and the state variable in the nonlinear term of the system. The structure of the newly constructed chaotic system is explored. When the number of the same state variables in the nonlinear term of the chaotic system varies, the system's dynamic behavior undergoes complex changes. Moreover, with the exponential change of a single-state variable in a three-dimensional system, the system maintains the chaotic attractor while the state of the attractor changes. On this basis, the Lyapunov exponent, bifurcation diagram, complexity, and 0-1 test are used to compare and analyze this phenomenon. Through circuit simulations, the chaotic characteristics of the system under different conditions are further verified; this provides a theoretical basis for the hardware implementation of the new system. Finally, the new chaotic system is applied to an image encryption system with the same encryption and decryption processes. The comparison shows improved encryption and decryption characteristics of image encryption systems.
Theory of reduced biquaternion sparse representation and its applications

Traditional sparse representation models treat color image either represent color channels independently using the monochromatic model or concatenate color channels using the concatenation model. However, these two strategies cannot make full use of the correlation among the three color channels. In this paper, we propose a novel sparse representation model for color image based on reduced biquaternion. The advances of the proposed model are in two aspects: 1) compared with quaternion algebra, reduced biquaternion algebra is commutative algebra which has a low computation cost; 2) benefited from reduced biquaternion representation, the proposed model can treat the color image as a whole. We firstly extend the singular value decomposition (SVD) to the reduced biquaternion domain (RBSVD). The framework of the proposed model contains two major stages: first, reduced biquaternion orthogonal matching pursuit (RBOMP) is proposed in the stage of sparse coding; second, generalized K-means clustering for RBSVD (K-RBSVD) is proposed in the stage of dictionary training. Afterward, a new color image denoising algorithm is developed to demonstrate the effectiveness of the proposed model. The experimental results demonstrate that the proposed sparse representation model achieves state-of-the-art denoising performance in terms of both quantitative metrics and visual quality.
Coal rock image recognition method based on improved CLBP and receptive field theory

Abstract Rapid coal‐rock identification is one of the key technologies for intelligent and unmanned coal mining. Currently, the existing image recognition algorithms cannot satisfy practical needs in terms of recognition speed and accuracy. In view of the evident differences between coal and rock in visual attributes such as color, gloss and texture, the complete local binary pattern (CLBP) image feature descriptor is introduced for coal and rock image recognition. Given that the original algorithm oversimplifies local texture features by ignoring imaging information from higher‐order pixels and the concave and convex areas between adjacent sampling points, this paper proposes a higher‐order differential median CLBP image feature descriptor to replace the original CLBP center pixel gray with a local gray median, and replace the binary differential with a second‐order differential. Meanwhile, for the high dimensionality of CLBP descriptor histogram and feature redundancy, deep learning perceptual field theory is introduced to realize data nonlinear dimensionality reduction and deep feature extraction. With relevant experiments conducted, the following conclusion can be drawn: (1) Compared with that of the original CLBP, the recognition accuracy of the improved CLBP algorithm is greatly improved and finally stabilized above 94.3% under strong noise interference; (2) Compared with that of the original CLBP model, the single image recognition time of the coal rock image recognition model fusing the improved CLBP and the receptive field theory is 0.0035 s, a reduction of 71.0%; compared with the improved CLBP model (without the fusion of receptive field theory), it can shorten the recognition time by 97.0%, but the accuracy rate still maintains more than 98.5%. The method offers a valuable technical reference for the fields of mineral development and deep mining.
A Single-Image Noise Estimation Algorithm Based on Pixel-Level Low-Rank Low-Texture Patch and Principal Component Analysis

Noise level is an important parameter for image denoising in many image-processing applications. We propose a noise estimation algorithm based on pixel-level low-rank, low-texture subblocks and principal component analysis for white Gaussian noise. First, an adaptive clustering algorithm, based on a dichotomy merge, adaptive pixel-level low-rank matrix construction method and a gradient covariance low-texture subblock selection method, is proposed to construct a pixel-level low-rank, low-texture subblock matrix. The adaptive clustering algorithm can improve the low-rank property of the constructed matrix and reduce the content of the image information in the eigenvalues of the matrix. Then, an eigenvalue selection method is proposed to eliminate matrix eigenvalues representing the image to avoid an inaccurate estimation of the noise level caused by using the minimum eigenvalue. The experimental results show that, compared with existing state-of-the-art methods, our proposed algorithm has, in most cases, the highest accuracy and robustness of noise level estimation for various scenarios with different noise levels, especially when the noise is high.
Recognizing Teachers’ Hand Gestures for Effective Non-Verbal Interaction

Hand gesturing is one of the most useful non-verbal behaviors in the classroom, and can help students activate multi-sensory channels to complement teachers’ verbal behaviors and ultimately enhance teaching effectiveness. The existing mainstream detection algorithms that can be used to recognize hand gestures suffered from low recognition accuracy under complex backgrounds and different backlight conditions. This study proposes an improved hand gesture recognition framework based on key point statistical transformation features. The proposed framework can effectively reduce the sensitivity of images to background and light conditions. We extracted key points of the image and establish a weak classifier to enhance the anti-interference ability of the algorithm in the case of noise and partial occlusion. Then, we used a deep convolutional neural network model with multi-scale feature fusion to recognize teachers’ hand gestures. A series of experiments were conducted on different human gesture datasets to verify the performance of the proposed framework. The results show that the framework proposed in this study has better detection and recognition rates compared to the you only look once (YOLO) algorithm, YOLOv3, and other counterpart algorithms. The proposed framework not only achieved 98.43%, measured by F1 score, for human gesture images in low-light conditions, but also has good robustness in complex lighting environments. We used the proposed framework to recognize teacher gestures in a case classroom setting, and found that the proposed framework outperformed YOLO and YOLOv3 algorithms on small gesture images with respect to recognition performance and robustness.
Variational Quantum Algorithm Applied to Collision Avoidance of Unmanned Aerial Vehicles

Mission planning for multiple unmanned aerial vehicles (UAVs) is a complex problem that is expected to be solved by quantum computing. With the increasing application of UAVs, the demand for efficient conflict management strategies to ensure airspace safety continues to increase. In the era of noisy intermediate-scale quantum (NISQ) devices, variational quantum algorithms (VQA) for optimizing parameterized quantum circuits with the help of classical optimizers are currently one of the most promising strategies to gain quantum advantage. In this paper, we propose a mathematical model for the UAV collision avoidance problem that maps the collision avoidance problem to a quadratic unconstrained binary optimization (QUBO) problem. The problem is formulated as an Ising Hamiltonian, then the ground state is solved using two kinds of VQAs: the variational quantum eigensolver (VQE) and the quantum approximate optimization algorithm (QAOA). We select conditional value-at-risk (CVaR) to further promote the performance of our model. Four examples are given to validate that with our method the probability of obtaining a feasible solution can exceed 90% based on appropriate parameters, and our method can enhance the efficiency of a UAVs' collision avoidance model.
Application of deep learning for seismicity analysis in Ghana

We present the characterization of regional seismicity in Ghana by processing the Ghana Digital Seismic Network (GHDSN) data set recorded between September 2012 and April 2014, implementing deep learning (DL). Local earthquakes are detected in this dataset using EQTransformer, a DL model with a hierarchical attentive mechanism (HAM) for simultaneous earthquake detection and P- and S-phase picking. A Conservative Strategy (CS) is devised to detect the missing phases and to associate the detected phases to circumvent the false-negative issue of EQTrans-former processing low signal-to-noise ratio (SNR) seismograms. We performed a joint inversion by grid search in 1D velocity model space and simultaneous inversion for the hypocentral parameters, incorporating 559 detected arrival times (292 P and 267 S phases). The results obtained by velocity inversion contain thicknesses of 1, 13, 8, 13, and 10 km, from the surface to a depth of 45 km, with Vp = 5.9, 6.1, 6.3, 6.5, 6.9, and 7.2 km/s, respectively. The updated velocities for the first and last layers are 6% and 26% and the Vp/Vs (=1.70) is 3.03% higher than the previously reported values. A total number of 73 earthquakes with a local magnitude of 2.5 < Ml < 3.9 are located, comprising four main clusters of events, showing a high correlation with the mapped faults zones. The hypocentral depth distribution is mainly in the range of 7-15 km, confined to the upper crust in the region. No specific seismic activity in the eastern branch of Coastal Boundary Fault (CBF) and the continuation of Romanche Fracture Zone (RFZ) in the study period was observed, casting further doubt on the activity of this branch and the hypothesis of stress transfer from RFZ to southern Ghana. The results reinforce the intraplate nature of the tectonic activities in the region. Finally, an updated seismic catalog up to April 2022 is presented for Ghana by incorporating all reported catalogs and combining the newly detected events.
Reversible data hiding for color images based on pixel value order of overall process channel

Color image Reversible Data Hiding (RDH) is getting more and more important since the number of its applications is steadily growing. This paper proposes an efficient color image RDH scheme based on pixel value ordering (PVO), in which the channel correlation is fully utilized to improve the embedding performance. In the proposed method, the channel correlation is used in the overall process of data embedding including prediction stage, block selection and capacity allocation. In the prediction stage, since the pixel values in the co-located blocks in different channels are monotonically consistent, the large pixel values are collected preferentially by pre-sorting the intra-block pixels. This can effectively improve the embedding capacity of RDH based on PVO. In the block selection stage, the description accuracy of block complexity value is improved by exploiting the texture similarity between the channels. The smoothing the block is then preferentially used to reduce invalid shifts. To achieve low complexity and high accuracy in capacity allocation, the proportion of the expanded prediction error to the total expanded prediction error in each channel is calculated during the capacity allocation process. The experimental results show that the proposed scheme achieves significant superiority in fidelity over a series of state-of-the-art schemes. For example, the PSNR of the Lena image reaches 62.43 dB, which is a 0.16 dB gain compared to the best results in the literature with a 20,000bits embedding capacity.
Single image deraining using multi-scales context information and attention network

The existing deraining methods based on convolutional neural networks (CNNs) have made great success, but some remaining rain streaks can degrade images drastically. In this work, we proposed an end-to-end multi-scale context information and attention network, called MSCIANet. The proposed network consists of multi-scale feature extraction (MSFE) and multi-receptive fields feature extraction (MRFFE). Firstly, the MSFE can pick up features of rain streaks in different scales and propagate deep features of the two layers across stages by skip connections. Secondly, the MRFFE can refine details of the background by attention mechanism and the depthwise separable convolution of different receptive fields with different scales. Finally, the fusion of these outputs of two subnetworks can reconstruct the clean background image. Extensive experimental results have shown that the proposed network achieves a good effect on the deraining task on synthetic and real-world datasets. The demo can be available at https://github.com/CoderLi365/MSCIANet.