Reviewerly/exHarmony huggingface dataset:
Efficiently Amalgamated CNN-Transformer Network for Image Super-Resolution Reconstruction

Currently, heavy and sophisticated neural network models are designed to improve image super-resolution reconstruction accuracy. However, the model requires high computation resources and is difficult to deploy on mobile devices. Therefore, designing an efficient lightweight image super-resolution network is urgent. We propose to apply the self-attention mechanism in transformer to compensate for the limitations of convolutional neural networks in global representation. In particular, we apply the split depth-wise transposed orthogonal transformer encoder, which splits the input tensor into feature subsets, and depth-wise separable convolution and self-attention across channel dimensions to implicitly increase the receptive field and encode multi-scale features. Experimental results confirmed that our proposed lightweight image super-resolution network (ACTNet) based on an amalgamate CNN-Transformer can achieve better performance than state-of-the-art methods.
CSTrack: A Comprehensive and Concise Vision Transformer Tracker

The attention mechanism has been widely applied in visual tracking tasks due to its remarkable ability to capture global dependencies. However, there are two issues in previous attention-based methods: redundancy of template information and inappropriate utilization of information streams. In this work, we propose a spatial positioning attention mechanism that addresses these issues by selective template feature enhancement and elimination of redundant information streams, respectively, significantly improving tracking accuracy and speed. Furthermore, previous trackers fail to focus on channels containing crucial target information within the template features and search region features. To tackle this, we introduce a channel focus attention mechanism to perform channel weight rescaling, which allows the tracker to concentrate on those target-related channels, improving its localization capability. Extensive experiments on four well-known datasets, GOT-10k, LaSOT, TrackingNet, and TNL2K, show that CSTrack outperforms all previous state-of-the-art trackers, running at over 70 FPS.
Semantic-Guided Completion Network for Video Inpainting in Complex Urban Scene

Video inpainting aims to fill damaged areas in video frames with appropriate content. Complex scene contain cluttered or ambiguous semantics and objects, making video inpainting in such scenarios a challenging yet meaningful task. Current methods are limited by the lack of sufficient video information, resulting in blurred results and temporal artifacts. In this paper, we design a novel semantic-guided completion network that uses the semantic information of the videos to complete missing regions in the complex urban scene. Specifically, we first leverage the semantic information to model the structure and content of the video and improve U-Net network to complete the broken semantic image. Then, we propose a module based on spatial-adaptive normalization to guide the generation of the damaged part of the video pixels by combining semantic information. Our model’s ability to generate reasonable and accurate content is demonstrated through both quantitative and qualitative results on two publicly available urban scene datasets.
OKGR: Occluded Keypoint Generation and Refinement for 3D Object Detection

Lidar-based 3D object detectors utilize point clouds to detect objects in autonomous driving. However, the point clouds are sparse and incomplete, which affects the detectors’ learning of shape knowledge and limits the 3D detection performance. Previous works improve performance through completing object shape at the point level or representation level, such as voxel. The former increases computational burden, while the latter has poor generalization ability to point-based detectors. In this paper, we present an approach, namely Occluded Keypoint Generation and Refinement (OKGR), which is effective to improve 3D detection performance by completing object features at the keypoint level. Specifically, Occluded Keypoint Generation (OKG) generates occluded keypoints to densify raw keypoints and learns the offsets between the generated keypoints and prototypes, while retaining the raw keypoints unchanged. Occluded Keypoint Refinement (OKR) assigns weights to the generated keypoints and conducts these weights to features to obtain high-quality complete features for detection. We apply our approach to two representative detectors, PV-RCNN++ and PDV, and evaluate the detectors on KITTI and Waymo Open Dataset. The experiments show significant performance improvement. Particularly, our OKGR applied on PV-RCNN++ achieves improvements of Pedestrian and Cyclist of +3.19%, +2.53% AP on average difficulty levels on KITTI, and +2.18%, +2.29% mAPH on Waymo Open Dataset. For more information, the supplementary material and code are available at https://github.com/Mingqj/OKGR .
Just‐in‐time identification for cross‐project correlated issues

Abstract Issue tracking systems are now prevalent in software development, which would help developers submit and discuss issues to solve development problems on software projects. Most previous studies have been conducted to analyze issue relations within projects, such as recommending similar or duplicate bug issues. However, along with the popularization of co‐developing through multiple projects, many issues are cross‐project correlated (CPC), that is, one issue is associated with another issue in a different project. When developers meet with CPC issues, it may primarily increase the difficulties of solving them because they need information from not only their projects but also other related projects that developers are not familiar with. Identifying a CPC issue as early as possible is a fundamental challenge for both managers and developers to allocate the resources for software maintenance and estimate the effort to solve it. This paper proposes 11 issue metrics of two groups to describe textual summary and reporters' activity, which can be extracted just after the issue was reported. We employ these 11 issue metrics to construct just‐in‐time (JIT) prediction models to identify CPC issues. To evaluate the effect of CPC issue prediction models, we conduct experiments on 16 open‐source data science and deep learning projects and compare our prediction model with two baseline models based on textual features (i.e., Term Frequency‐Inverse Document Frequency [TF‐IDF] and Word Embedding), which are commonly adopted by previous studies on issue prediction. The results show that the JIT prediction model based on issue metrics has significantly improved the performance of CPC issue prediction under two evaluation indicators, Matthew's correlation coefficient (MCC) and F1. In addition, we find that the prediction model is more suitable for large‐scale complex core projects in the open‐source ecosystem.
A Real-Time Safety Detector Based on Re-parameterization Multiscale Feature Fusion for Forklift Driving

The application of object detection in intelligent logistics has received considerable attention. However, existing detector models face challenges such as high computational costs, slow detection speed, and difficulty in deployment on edge devices with limited computational resources. This paper proposes a novel real-time safety detector model (RMFFDet) based on YOLOv8s for forklift driving. A hardware-friendly FasterNeXt module is designed to optimize feature extraction and reduce the computational costs in the Backbone. Inspired by the work on RepGhost, a re-parameterization multiscale feature fusion Neck (RMFFNeck) is proposed in this paper. Reconstructing the Neck based on RMFFNeck improves the capture of contextual logistics background feature information while reducing the model parameters. Finally, the Wise-IoU (WIoU) is introduced as a bounding box regression loss combined with a dynamic non-monotonic focusing mechanism to improve the model’s overall performance. Experiments show that RMFFDet achieves a mean Average Precision (mAP) of 95.2% on the KITTI dataset and 92.8% on the self-built Forklift-3k dataset. Compared to YOLOv8s, the model parameters are reduced by 34.5%. On the Jetson Nano edge platform and 640 $$\times $$ 640 input size, RMFFDet requires only 100.2ms inference time. RMFFDet offers an excellent trade-off between inference speed and detection accuracy. It meets the industrial requirements of logistics scenarios.
Enhancing Image-to-Image Translation with Contrast Loss Constrained Generators and Selective Neighborhood Sampling

Unpaired image-to-image (I2I) translation has been a popular research topic in the field of computer vision, and researchers have recently obtained the desired images by introducing contrastive loss to constrain the generative process of generators. However, when constructing negative samples for contrastive learning, they use a random sampling strategy, which may affect the performance of the model. In this paper, we address this issue by deliberately selecting image blocks with high overlapping semantic information. We designed an SNS module to compute a similarity matrix by comparing the feature distances between the original image blocks and the generated images, each row of this matrix corresponds to probability distribution between the image patches, and we select the few positions with the highest probability for sampling. In addition, we note that the domain invariant information in the generated images is subject to problems such as distortion and discoloration, and we use the attention mechanism to separate the domain invariant information and use this information as the final result of the auxiliary computation. We validate our proposed method on three different image translation datasets, showing that it improves the quality of the generated images without adding additional learnable parameters.
Machinery Fault Signal Detection with Deep One-Class Classification

Fault detection of machinery systems is a fundamental prerequisite to implementing condition-based maintenance, which is the most eminent manufacturing equipment system management strategy. To build the fault detection model, one-class classification algorithms have been used, which construct the decision boundary only using normal class. For more accurate one-class classification, signal data have been used recently because the signal data directly reflect the condition of the machinery system. To analyze the machinery condition effectively with the signal data, features of signals should be extracted, and then, the one-class classifier is constructed with the features. However, features separately extracted from one-class classification might not be optimized for the fault detection tasks, and thus, it leads to unsatisfactory performance. To address this problem, deep one-class classification methods can be used because the neural network structures can generate the features specialized to fault detection tasks through the end-to-end learning manner. In this study, we conducted a comprehensive experimental study with various fault signal datasets. The experimental results demonstrated that the deep support vector data description model, which is one of the most prominent deep one-class classification methods, outperforms its competitors and traditional methods.
HPAN: A Hybrid Pose Attention Network for Person Re-Identification

To address the difficulty in expressing the correlation between different local features extracted by the current person re-identification feature extraction methods, and the challenge of effectively integrating local features with global features, a Hybrid Pose Attention Network (HPAN) for person re-identification is proposed. In HPAN, the high-resolution network HRNet-W32 serves as the backbone for person re-identification and pose estimation, extracting global features and local key point heatmaps of the human images, and then generating local key point features. Self-attention is used to extract the correlation between each local key point feature, generating local pose features. Furthermore, a Hybrid Pose and Global Feature Fusion (HPGFF) module is adopted to fuse the global features and local pose features, creating integrated features. To evaluate, we conduct experiments on five publicly available datasets, and HPAN has all achieved competitive or state-of-the-art results.
Feature Disentanglement and Adaptive Fusion for Improving Multi-modal Tracking

Multi-modal tracking has increasingly gained attention due to its superior accuracy and robustness in complex scenarios. The primary challenges in this field lie in effectively extracting and fusing multi-modal data that inherently contain gaps. To address the above issues, we propose a novel regularized single-stream multi-modal tracking framework, drawing inspiration from the perspective of disentanglement. Specifically, taking into account the similarities and differences intrinsic in multi-modal data, we design a modality-specific weights sharing feature extraction module to extract well-disentangled multi-modal features. To emphasize feature-level specificity across different modal features, we propose a cross-modal deformable attention mechanism for the adaptive integration of multi-modal features with efficiency. Through extensive experiments on three multi-modal tracking benchmarks, including RGB+Thermal infrared and RGB+Depth, we demonstrate that our method significantly outperforms existing multi-modal tracking algorithms. Code is available at https://github.com/ccccwb/Multimodal-Detection-and-Tracking-UAV .
Creating a software prototype of the face recognition system for educational institutions on the "1C:Enterprise" platform

The use of information technology entails not only huge benefits but also a large number of new threats. One of the most dangerous of them is data leaks, as they undermine consumer confidence and can lead to business shutdowns. Educational organizations that use applications on the 1C platform in their work face a similar problem. The solution in this case may be to improve the authorization process by adding one more step to it — face recognition. For optimal design of this authorization method, it is necessary to develop the program architecture considering the current business requirements of educational organizations and to develop a software prototype ready for integration into current educational software solutions. The article provides an example of the above architecture and the application created on its basis. Further, they can be integrated into any system based on “1C:Enterprise”. To meet the requirements for the system, the most appropriate architecture is a “client — server” with a separate server dedicated to the database. Python programming language is used, network communication is organized according to REST principles using GET method and HTTP protocol, photo data transfer is in JSON format.
Secure Collaborative Computing for Linear Regression

Machine learning usually requires a large amount of training data to build useful models. We exploit the mathematical structure of linear regression to develop a secure and privacy-preserving method that allows multiple parties to collaboratively compute optimal model parameters without requiring the parties to share their raw data. The new approach also allows for efficient deletion of the data of users who want to leave the group and who wish to have their data deleted. Since the data remain confidential during both the learning and unlearning processes, data owners are more inclined to share the datasets they collect to improve the models, ultimately benefiting all participants. The proposed collaborative computation of linear regression models does not require a trusted third party, thereby avoiding the difficulty of building a robust trust system in the current Internet environment. The proposed scheme does not require encryption to keep the data secret, nor does it require the use of transformations to hide the real data. Instead, our scheme sends only the aggregated data to build a collaborative learning scheme. This makes the scheme more computationally efficient. Currently, almost all homomorphic encryption schemes that support both addition and multiplication operations demand significant computational resources and can only offer computational security. We prove that a malicious party lacks sufficient information to deduce the precise values of another party’s original data, thereby preserving the privacy and security of the data exchanges. We also show that the new linear regression learning scheme can be updated incrementally. New datasets can be easily incorporated into the system, and specific data can be removed to refine the linear regression model without the need to recompute from the beginning.
Fine-Grained Radio Frequency Fingerprint Recognition Network Based on Attention Mechanism

With the rapid development of the internet of things (IoT), hundreds of millions of IoT devices, such as smart home appliances, intelligent-connected vehicles, and wearable devices, have been connected to the network. The open nature of IoT makes it vulnerable to cybersecurity threats. Traditional cryptography-based encryption methods are not suitable for IoT due to their complexity and high communication overhead requirements. By contrast, RF-fingerprint-based recognition is promising because it is rooted in the inherent non-reproducible hardware defects of the transmitter. However, it still faces the challenges of low inter-class variation and large intra-class variation among RF fingerprints. Inspired by fine-grained recognition in computer vision, we propose a fine-grained RF fingerprint recognition network (FGRFNet) in this article. The network consists of a top-down feature pathway hierarchy to generate pyramidal features, attention modules to locate discriminative regions, and a fusion module to adaptively integrate features from different scales. Experiments demonstrate that the proposed FGRFNet achieves recognition accuracies of 89.8% on 100 ADS-B devices, 99.5% on 54 Zigbee devices, and 83.0% on 25 LoRa devices.
MFM: A Multiple-Features Model for Leisure Event Recommendation in Geotagged Social Networks

Event-based social networks (EBSNs) are rich in information about users and leisure events. The willingness of users to participate in leisure events is influenced by many factors such as event time, location, content, organizer, and social relationship factors of users. Event recommendation systems in EBSNs can help leisure event organizers to accurately find users who want to participate in events. However, to address the existing cold-start problems and improve the accuracy of event recommendations, we propose a multiple-feature-based leisure event recommendation model (MFM). We introduce the user’s social contacts into the user preference features and construct a user feature space by integrating the features of the user preferences for events and organizers and preferences of the user’s closest friends. Moreover, considering the behavioral differences between active and inactive users, we extracted the respective features and trained the feature weight models. Finally, the experimental results showed that in comparison with the baseline models, the precision of the MFM is higher by at least 7.9%.
MultiFusedNet: A Multi-Feature Fused Network of Pretrained Vision Models via Keyframes for Student Behavior Classification

This research proposes a deep learning method for classifying student behavior in classrooms that follow the professional learning community teaching approach. We collected data on five student activities: hand-raising, interacting, sitting, turning around, and writing. We used the sum of absolute differences (SAD) in the LUV color space to detect scene changes. The K-means algorithm was then applied to select keyframes using the computed SAD. Next, we extracted features using multiple pretrained deep learning models from the convolutional neural network family. The pretrained models considered were InceptionV3, ResNet50V2, VGG16, and EfficientNetB7. We leveraged feature fusion, incorporating optical flow features and data augmentation techniques, to increase the necessary spatial features of selected keyframes. Finally, we classified the students’ behavior using a deep sequence model based on the bidirectional long short-term memory network with an attention mechanism (BiLSTM-AT). The proposed method with the BiLSTM-AT model can recognize behaviors from our dataset with high accuracy, precision, recall, and F1-scores of 0.97, 0.97, and 0.97, respectively. The overall accuracy was 96.67%. This high efficiency demonstrates the potential of the proposed method for classifying student behavior in classrooms.
Effectiveness of participatory digital training based on virtual classrooms in developing teaching skills

The study aimed to assess the effectiveness of digital participatory training based on virtual classrooms in developing teaching skills among pre-service teachers. A quasi-experimental approach was employed using a teaching skills checklist as the measurement tool. The sample included 46 students from the Child Education Department, intentionally divided into experimental (22) and control (24) groups. The findings indicate that participatory digital training based on virtual classrooms significantly enhanced the teaching skills and interaction among pre-service teachers. The study emphasised the importance of digital environments and emerging technologies such as virtual classrooms in pre-service teacher training. This approach has potential as an innovative solution for teacher training, particularly in light of social distancing measures and the impact of the COVID-19 pandemic on traditional learning. The researcher recommends embracing digital transformation in education by incorporating virtual classrooms as a global requirement to overcome the effects of prolonged global pandemics.
Double-Factor Tensor Cascaded-Rank Decomposition for Hyperspectral Image Denoising

Hyperspectral image (HSIs) denoising is a preprocessing step that plays a crucial role in many applications used in Earth observation missions. Low-rank tensor representation can be utilized to restore mixed-noise HSIs, such as those affected by mixed Gaussian, impulse, stripe, and deadline noises. Although there is a considerable body of research on spatial and spectral prior knowledge concerning subspace, the correlation between the spectral continuity and the nonlocal sparsity of the spectral and spatial factors is not yet fully understood. To address this deficiency, in the present study, we determined the correlation between these factors using a cascaded technique, and we describe in this paper the double-factor tensor cascaded-rank (DFTCR) minimization method that was used. The information existing in the nonlocal sparsity property of the spatial factor was employed to promote a geometrical feature representation, and a tensor cascaded-rank minimization approach was introduced as a nonlocal self-similarity to promote restoration quality. The continuity between the difference and nonlocal gradient sparsity constraints of the spectral factor was also introduced to learn the basis. Furthermore, to estimate the solutions of the proposed model, we developed an algorithm based on the alternating direction method of multipliers (ADMM). The performance of the DFTCR method was tested by a comparison with eleven established denoising methods for HSIs. The results showed that the proposed DFTCR method exhibited superior performance in the removal of mixed noise from HSIs.