Reviewerly/exHarmony huggingface dataset:
Safety Helmet Wearing Detection Based on Particle Swarm Optimization YOLOv7

In high-risk work environments, such as construction sites, the use of safety helmets is a mandatory safety measure to prevent head injuries. In this research paper, we propose a novel method for detecting the wearing of safety helmets using a combination of Particle Swarm Optimization (PSO) and YOLOv7, a state-of-the-art object detection algorithm. The aim of this method is to address the issue of detecting safety helmet wearing in high-risk work environments. The proposed approach optimizes the hyperparameters of YOLOv7 training using the PSO algorithm, which results in improved accuracy remarkably. To evaluate the proposed method, a dataset of real-world images is used, and it achieved high accuracy, outperforming other state-of-the-art methods. The results demonstrate the potential of the proposed method as a tool for enhancing safety in high-risk work environments by accurately detecting safety helmet wearing.
Multi-Level Evidence Active Learning for Water Surface Object Detection

Object detection is a rapidly evolving field, but there remains a challenge that the requirement of a significant amount of labeled data. In the recent five years, several active learning strategies have been proposed to achieve improved performance with reduced annotation cost on object detection tasks. However, these approaches have some limitations, including the incomplete utilization of epistemic uncertainty and bounding box information. To address these limitations, this paper proposes a novel active learning method named Multi-Level Evidence Active Learning (MLEAL) for water surface object detection. We propose to utilize Evidence Deep Learning (EDL) for uncertainty estimation and designed an Evidence Optimization Head (EOH) to adapt EDL for water surface object detection. To compute the information of an image by considering the multi-level information of the bounding box, we introduce to the utilization of Multi-Level Uncertainty Aggregation (MLUA) to represent the information over an image. The proposed MLEAL is valid on the Water Surface Object Detection Dataset (WSODD). The results demonstrate that the MLEAL outperforms existing active learning methods.
A Deep Knowledge Tracking Model Based on Forgetting Features and Self-Attention Mechanism

The integration of large education and artificial intelligence technologies is gradually deepening, and how to provide personalized user profiling services for learners is an important research problem. In response to the long-term dependency problem and lack of learning features of today’s deep knowledge tracking models, a deep knowledge tracking model (FSA-MuLSTM) based on forgetting features and the self-attentiveness mechanism is proposed. Firstly, the model combines the learner’s historical learning interaction tuple sequences with learning times and memory residuals to obtain the input vector based on the LSTM; secondly, the correlation weights between tuples are captured using the self-attention mechanism; finally, the deep information between tuple sequences is captured using the stacking of multilayer LSTMs. The probability of correctly answering the current question is derived from the hidden vectors and the self-attentive weights of the output of the multilayer LSTM. Two public datasets are used for the experiments, and the combined analysis outperforms existing knowledge-tracking models in terms of performance and improves the accuracy of prediction.
Gaussian Transformer and CNN Segmentation Method Based on Contrastive Learning of Boundary

Convolutional neural networks (CNNs) are the most widely used deep learning-based architecture for medical image analysis. However, the limited receptive field of existing CNN-based methods can result in inaccurate segmentation, especially in cases of partial similarity or local environmental changes. To address this issue, transformer architectures have been introduced as an effective solution. However, self-attention mechanism used in the transformer considers all features through weighted averaging, which limits its ability to capture local dependencies. While this may not be a significant issue for internal regions of lesions, accurately delineating small lesion boundaries can be challenging. This is because local correlation is often more critical than distant correlation in medical image segmentation tasks. Furthermore, the precise segmentation of breast tumors in ultrasound images remains challenging due to unclear boundaries. In this study, we propose an angular-margin contrastive learning model that integrates information from gaussian transformer and CNN decoder. Our model uses a shallow CNN encoder, followed by a CNN decoder and a gaussian transformer decoder. Features learned from both branches are then fused for joint prediction. To maintain the U-shape structure between the decoder and encoder, each decoding feature map is skip-connected to its corresponding encoder feature. We introduce a gaussian weighting in traditional multi-head attention to enhance the local correlation modeling ability of self-attention. Additionally, we propose an angular-margin contrastive loss to encourage the model to focus on boundary features and improve segmentation performance. The proposed method is validated on an ABUS dataset, and experimental results demonstrate its state-of-the-art performance, with a Dice similarity coefficient and HD95 of 0.813 and 2.207 mm, respectively. It is worth noting that our method considers not only global semantics but also local correlation and boundary features, thereby improving segmentation accuracy.
Progressive Feature Fusing for High-Resolution Image Inpainting

The purpose of image inpainting is to generate clear and realistic images. Despite the rapid development of image inpainting technology, there remain issues such as unclear texture of images and a lack of interaction between structure inpainting and texture inpainting. As a result, when dealing with large missing regions, the generated images are generally distorted. To address these problems, we propose a two-stage image inpainting network, which aims to achieve appropriate filling content through a coarse inpainting network, and subsequently enhance the content image details with a fine inpainting network. Moreover, we introduce a global and local gated convolution network (GLGC) module that utilizes long-range context information and thus performs better than the widely-used vanilla convolutions in balancing the structure and texture of images. We conduct experiments on two public datasets: CelebA-HQ and Paris StreetView. The results demonstrate that the proposed two-stage image inpainting network using our GLGC module enhances the quality of generated images on datasets with different scenes.
MMST-GCN+: A Multi-Modal ST-GCN+ for Computer-Aided Education by Introducing a Sensor Skeleton

Multi-model data can improve the performance of image-based models for computer-aided education (CAE) by utilizing features from more dimensions rather than images only. The spatial temporal graph convolution network (ST-GCN) is a two-stream model for classifying human actions based on their skeleton data. This work proposes a novel intelligent data model (MMST-GCN) based on ST-GCN for multi-model image-based CAE. MMST-GCN integrates image and sensor data into a unified framework by introducing a sensor skeleton that provides stronger feature representation and classification abilities. Moreover, a newly modified network structure (ST-GCN+) is developed to percept the slow actions more accurately, promoting the classification performance for representative actions occurring during CAE analysis. MMST-GCN+ is evaluated on two multimodal human action datasets (MHADs) and one homemade teacher action dataset constructed by investigating the research focus on CAE. It achieves better results of 96.21%, 96.33%, 96.21% on the accuracy, macro F1-score and micro F1-score, respectively, on the Berkeley-MHAD, 91.55%, 91.93%, 91.55% of the corresponding indices on the UTD-MHAD and 79.18%, 80.38%, 79.18% on the same indicators for the self-made dataset Teacher-MHAD.
Research on Image Reconstruction Method Based on Generative Adversarial Network

The traditional SRGAN image super-resolution reconstruction algorithm has a poor reconstruction quality and does not consider the detailed information at different scales. To address these issues, improvements were made to the generation network and discrimination network of the traditional SRGAN model. Channel attention mechanism and spatial attention mechanism were introduced into the residual block and upsampling module of the generation network to enhance the network's perceptual ability and reconstruction quality. In addition, a pyramid structure was introduced into the discriminator, allowing it to simultaneously consider the detailed information at different scales and better understand the spatial structure of the entire image. Test results on the Set5, Set14, and Urban100 datasets show that the improved SRGAN algorithm has better PSNR and SSIM than the traditional SRGAN algorithm. Furthermore, the improved SRGAN algorithm can better reconstruct the detailed parts of the image, making the reconstructed image clearer.
A Novel Self-Adaptive Vehicle Segmentation Method in Traffic Video with Symmetric Frame Difference Constraints

Vehicle target detection technology for traffic video is the technical foundation and key bottleneck technology of traffic monitoring. Aiming at the high-definition video image collected by the roadside monitoring equipment, this paper explores the vehicle detection method based on the traffic video. In the traffic scene, moving and stopping are the vehicle target participating states that affect the road safety. In this paper, a vehicle self-adaptive threshold segmentation method for traffic video under the constraint of symmetrical frame difference is proposed. By the idea of symmetrical difference, the adjacent three consecutive images are differentiated, the background of the high-definition monitoring video frame is extracted and updated, the moving vehicle target in the result image is segmented and detected by self-adaptive threshold, and the stopped vehicle is segmented by background updating and filtering. The experimental results show that the proposed method can segment vehicle targets with different traffic states completely and accurately. In addition, the proposed method effectively improve the detection accuracy of vehicle targets with different situations.
UAVS Task Assignment Based on Hybrid Swarm Intelligence Algorithm

Aiming at the contradiction between the globalism and the convergence speed of the UAVS task assignment method, a hybrid swarm intelligence algorithm is proposed, which combines the wolf colony algorithm and simulated annealing algorithm. Firstly, based on the analysis of UAV cluster operation application and task allocation modeling, the basic wolf colony algorithm is improved, the fixed step size of walking, summoning and rounding up behavior in the wolf colony algorithm is changed to the adaptive step size to improve the convergence speed and search accuracy; Secondly, the initial results of the improved wolf colony algorithm are taken as the input of the simulated annealing algorithm, and a fast search is carried out to further improve the convergence speed of the algorithm; Finally, in the same application scenario, through simulation and comparison with particle swarm optimization algorithm and basic wolf swarm optimization algorithm, it is verified that the proposed algorithm has great improvement in distribution accuracy and convergence speed.
A Survey of GNN-Based Graph Similarity Learning

With the advancement of deep learning, the application of graphs has become increasingly widespread. Graph similarity learning has gained significant attention as a crucial component in various learning tasks, such as classification, clustering, and subgraph matching, making it a research hotspot in the field of graphs. Currently, graph similarity learning based on graph neural networks (GNNs) primarily involves mapping input graphs to a target space using deep learning models, aiming to approximate the structural distances between graphs in the output space. Firstly, this paper provided a comprehensive review of research on GNN-based graph similarity learning. Based on different graph representation learning approaches, it categorized them into three types: GNN-CNN hybrid models for graph similarity learning, Siamese GNN-based graph similarity learning, and GNN-based graph matching networks for similarity learning. Secondly this paper then provided a detailed analysis of these three types of models. Finally, it discusses the challenges and future research directions in GNN-based graph similarity learning.
Enhancing Continuous Sign Language Recognition with Self-Attention and MediaPipe Holistic

Sign language recognition (SLR) is an interdisciplinary application that combines computer vision and natural language processing. This paper focuses on Continuous sign language recognition (CSLR) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> , which refers to recognizing a continuous sequence of sign language sentences, phrases, or words expressed in a short video. Currently, most phrase-level CSLR research primarily uses recurrent neural networks (RNNs). However, RNNs struggle to capture global dependencies and can only model sequential actions. Sign language involves complex spatial patterns formed by hand gestures, facial expressions, and body movements. Therefore, the global dependency of the spatial features extracted from the video is crucial for this task. In this paper, we introduce a novel pipeline framework for addressing CSLR. We first employ MediaPipe Holistic to extract key points from sign language videos, which are converted into sequential input data that the model can process. To overcome the disadvantages of RNNs, we use the Self-Attention mechanism, which excels at identifying relationships among key points and capturing global dependencies between sign language actions within a sequence. Combining the Self-Attention model with the extracted key points creates a more effective and efficient solution for CSLR. Additionally, we discuss the selection of MediaPipe Holistic key points, as not all key points equally contribute to the recognition. Experimental results show that the proposed pipeline exhibits promising performance on the first ten glosses (classes) of the Word-Level American Sign Language (WLASL-10) dataset.
VGGAN: Visual Grounding GAN Using Panoptic Transformers

Visual Grounding is an important part of image annotation generation. The existing methods usually use data alignment based on the similarity calculation of visual text features in location inference and multi-modal fusion, which will lose visual and text information to some extent, and is more likely to make the model overfit the data of specific scenes. To solve this problem, we propose a Visual Grounding Generative Adversarial Network (VGGAN) for visual text fusion using the panoptic transformer. We use the generative adversarial network to generate the prediction, judge the accuracy, and design the visual text transformer according to the panoptic theory. The model can retain the feature information, realize the full interactions between features, thereby better supporting the feature fusion of visual and text. Experimental results on the COCO dataset of complex daily scenes verify the effectiveness of our model, and our model achieves the highest prediction accuracy compared with the state-of-the-art methods.
Virtual Viewpoint Rendering Method Based on Multi-Threshold Layering

In order to solve the problem of artifacts and holes in virtual viewpoint rendering, this paper proposes a virtual viewpoint rendering method based on multi-threshold layering. The processing flow of the method is divided into two algorithms, the pre-algorithm solves the artifact problem, and the post-algorithm solves the hole problem. In the pre-algorithm, the depth refinement and pixel correction parts produce precise pixel alignment between color and depth image pairs, eliminating contour artifacts, and then the color correction section eliminates color aberration artifacts. In the post-algorithm, the Otsu multi-threshold layering part separates the scene by layer, and combines with selected median filtering to eliminate crack holes. Then, the depth-guided hole filling part fills in occlusion holes and out-of-field holes based on pixel categories, improving the quality of the virtual view. Experiments have shown that our method performs better than many other methods in terms of subjective visual quality and objective evaluation.
SF-YOLO: RGB-T Fusion Object Detection in UAV Scenes

In recent years, unmanned aerial vehicles (UAV) have been widely used in the field of object detection due to their high flexibility, strong maneuverability, and ability to carry multiple types of cameras. With the development of RGB-Thermal (RGB-T) perception technology and the increasing demand for military use, a network structure that fuses RGB and thermal images with low false alarm rate and high detection precision has become a research focus. When visible-light cameras are affected by lighting conditions (such as low light and foggy weather), thermal image information provides useful complementary information. However, how to effectively fuse RGB and thermal images is still a challenge. Previous works have involved some simple fusion strategies, such as merging them at the input or connecting multimodal data features inside the model, or applying attention to each data modality. These fusion strategies are direct but not sufficient. To address this issue, we propose a multimodal data fusion target detection network called SF-YOLO. We develop an attention interaction enhanced fusion module (AIEF) to fully utilize complementary information while reducing the impact of redundant information. In addition, to address the problem of weak feature extraction ability in the CNN network structure, we introduce the Swin-Transformer module, and for the problem of large-scale variations in target size in UAV aerial images, we design a shallow enhanced feature pyramid (SEFP). The SF-YOLO multimodal data object detection algorithm proposed in this paper is tested on public datasets and our own dataset through a series of experiments, and the results show the effectiveness of our proposed method. This architecture is expected to be applied in military reconnaissance and public safety fields.
How to Explore the CV Screening Spatial Location: Towards an Eye Movement Experiment

The main purpose of this research is to statistically analysis the method of resume screening results and correlation between resume format, at the same time to explore the attention of resume format. The study was conducted from the perspective of psychology, introducing the point of eye movement research method, the attention in the process of resume screening and reading resume browsing rule, finally providing some advice and help for college graduates. Finally, through the results from eye movements, the best resume template is obtained, that is, work experience in C, school experience in D, winning experience in E area.
Aerial Image Stitching Using IMU Data from a UAV

Unmanned Aerial Vehicles (UAVs) are widely used for aerial photography and remote sensing applications. One of the main challenges is to stitch together multiple images into a single high-resolution image that covers a large area. Feature-based image stitching algorithms are commonly used but can suffer from errors and ambiguities in feature detection and matching. To address this, several approaches have been proposed, including using bundle adjustment techniques or direct image alignment. In this paper, we present a novel method that uses a combination of IMU data and computer vision techniques for stitching images captured by a UAV. Our method involves several steps such as estimating the displacement and rotation of the UAV between consecutive images, correcting for perspective distortion, and computing a homography matrix. We then use a standard image stitching algorithm to align and blend the images together. Our proposed method leverages the additional information provided by the IMU data, corrects for various sources of distortion, and can be easily integrated into existing UAV workflows. Our experiments demonstrate the effectiveness and robustness of our method, outperforming some of the existing feature-based image stitching algorithms in terms of accuracy and reliability, particularly in challenging scenarios such as large displacements, rotations, and variations in camera pose.
American Sign Language Alphabet Recognition with YOLOv5 Enhanced by MediaPipe Hands

Sign language recognition (SLR) aims to break the communication barrier between hearing-impaired individuals and others, and is beneficial in building an inclusive and caring society. This paper aims to achieve real-time static sign language interpretation from images employing deep learning methods. Conventionally, a pipeline approach is used to locate the position of the hand gesture in an image and then classify the gesture. You Only Look Once version 5 (YOLOv5) is a suitable model that captures the hand gesture with bounding boxes and predicts the gesture with a convolution-based classifier. However, in practice, SLR performance may be significantly affected when gestures are partially occluded, or the background environment is complicated. Therefore, this paper proposes a method using MediaPipe Hands to enhance hand features, enabling YOLOv5 to locate the hand position more precisely. In addition, MediaPipe Hands can predict finger position even when the hand is partly obscured, providing essential features in the classification stage. In experiments, MediaPipe Hands improves the success rate of hand localization in complex environments and increase hand gesture classification accuracy. Compared to the baseline model, the model employing features enhanced by MediaPipe Hands outperformed those without MediaPipe Hands in static sign language recognition (SSLR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> ). Moreover, our method was tested in real-life scenarios by implementing a web service application and demonstrated improved real-time recognition performance.