Reviewerly/exHarmony huggingface dataset:
ARU-DGAN: A dual generative adversarial network based on attention residual U-Net for magneto-acousto-electrical image denoising

Magneto-Acousto-Electrical Tomography (MAET) is a multi-physics coupling imaging modality that integrates the high resolution of ultrasound imaging with the high contrast of electrical impedance imaging. However, the quality of images obtained through this imaging technique can be easily compromised by environmental or experimental noise, thereby affecting the overall quality of the imaging results. Existing methods for magneto-acousto-electrical image denoising lack the capability to model local and global features of magneto-acousto-electrical images and are unable to extract the most relevant multi-scale contextual information to model the joint distribution of clean images and noise images. To address this issue, we propose a Dual Generative Adversarial Network based on Attention Residual U-Net (ARU-DGAN) for magneto-acousto-electrical image denoising. Specifically, our model approximates the joint distribution of magneto-acousto-electrical clean and noisy images from two perspectives: noise removal and noise generation. First, it transforms noisy images into clean ones through a denoiser; second, it converts clean images into noisy ones via a generator. Simultaneously, we design an Attention Residual U-Net (ARU) to serve as the backbone of the denoiser and generator in the Dual Generative Adversarial Network (DGAN). The ARU network adopts a residual mechanism and introduces a linear Self-Attention based on Cross-Normalization (CNorm-SA), which is proposed in this paper. This design allows the model to effectively extract the most relevant multi-scale contextual information while maintaining high resolution, thereby better modeling the local and global features of magneto-acousto-electrical images. Finally, extensive experiments on a real-world magneto-acousto-electrical image dataset constructed in this paper demonstrate significant improvements in preserving image details achieved by ARU-DGAN. Furthermore, compared to the state-of-the-art competitive methods, it exhibits a 0.3 dB increase in PSNR and an improvement of 0.47% in SSIM.
A trading matching model for aquatic products based on blockchain and credit mechanisms

Current online transactions of aquatic products are often plagued by problems such as low efficiency, high platform supervision cost, insufficient trust and leakage of transaction data. Blockchain has been widely used in many different fields due to its decentralization, non-tampering and distributed data management. In order to resolve the existing problems, a blockchain-based aquatic product trading matching model integrated with credit mechanisms is proposed in this study to improve the efficiency, quality, security and satisfaction of online transactions for aquatic products. Then, based on this model, an online trading matching prototype system for aquatic products is developed, taking the Hyperledger Fabric as the underlying architecture. The performance testing of the prototype system has demonstrated that the introduction of the credit mechanism has a certain improvement effect on the trading matching results of aquatic products, and the system can complete more than 1000 transactions within half an hour, which can satisfy the normal business-to-business online transaction needs for aquatic products. To a certain extent, it can reduce the security risks and supervision cost, and improve the efficiency and satisfaction of online transaction. This study can also bring insights to blockchain-based online trading models in other industry fields.
Synthesizing a Large Scene with Multiple NeRFs

Last several years, NeRF achieved great success in view synthesis since it can render high-quality images in a complex scene. However, we find that its ability to rebuild a large scene is low because images that are far apart and do not overlap with each other will affect each other in the training process. In order to solve this problem, we propose Cluster-based NeRF, which splits the original input images into several clusters and then train a NeRF for each cluster. We also design an algorithm to improve the rendering quality in the overlapping areas. In the experiments, we show that our method outperforms the traditional NeRF on both the blender and real world dataset.
Dense Visible Watermark Removal with Progressive Feature Propagation

Visible watermark removal is an important task to ensure the robustness of visible watermarks. However, existing techniques are unable to efficiently remove dense visible watermarks in stock content market images. To address this issue, we propose a novel solution called the Progressive Visible Watermark Removal Network (PVWRNet) that achieves efficient and effective dense visible watermark removal. Our approach utilizes a cross-combination structure in the backbone module to balance dense visible watermark spatial and contextual features. Furthermore, progressive modules are incorporated to multi-fuse shallow features of the backbone module, leading to significant improvement in image restoration metric scores and visual quality. To enhance local features of the backbone module, Reinforced Dual Attention Modules (RDAM) are introduced, which reweight the local features. Experimental results demonstrate that our proposed PVWRNet outperforms state-of-the-art approaches in the dense visible watermark dataset, especially in high definition images. In conclusion, the proposed method provides a potent and efficient solution for removing dense visible watermarks in stock content market images, and the use of cross-combination structure, progressive modules, and reinforced dual attention modules enables PVWRNet to outperform existing techniques in solving the challenging dense visible watermark removal problem. The code is available at: https://github.com/HyuuHyuuNeko/PVWRNet .
RoICLIP: Text-Enhanced UAV-Based Video Object Detection

In recent years, Unmanned Aerial Vehicles (UAV)-based video object detection algorithms have attracted a lot of attention due to their widespread applications in real life. However, due to the lack of visual features such as few pixels, motion blur, and defocus objects in the video, the accuracy improvement of existing video object detection methods on UAV-based videos is very limited. In this paper, we propose a novel video object detection framework RoICLIP, by transferring the knowledge from pre-trained CLIP models to the downstream video object detection task. Richer information can be obtained from text for frames in UAV videos with insufficient visual features. Specifically, a Cross-Modal Feature Enhancement(CMFE) module is designed to improve the transfer of pre-trained knowledge by establishing multi-modal correlations at different detection stages. It minimizes the gap between downstream and pre-training tasks, allowing the model to learn more comprehensive visual representations with text supervision. To further exploit temporal information within videos, we design an Attention-guided Frame-level Feature Aggregation(AFFA) module. Extensive experiments demonstrate the superior performance of our method on the task of UAV-based video object detection.
Multi-scale Non-local Bidirectional Fusion for Video Super-Resolution

Long-range dependency is one of the important inscriptions in sequence modeling. For video data, the commonly used convolutional and recurrent operations are a kind of “local coding” for variable-length sequences, which can only capture the local neighborhood information. We introduce the idea of non-local mean to compensate for the shortcomings of repeated convolutional operations, while most of the previous non-local methods used for video super-resolution only focus on positional information or fail to capture temporal information directly. In this study, we propose a non-local bidirectional fusion network (NLBF) for the video super-resolution (VSR) task. This non-local network decouples multidimensional information to reduce computational memory consumption, at the same time capturing long-range dependencies within the temporal-spatial-channel dimension as much as possible. In the multi-scale local and non-local hybrid framework, we further design the bidirectional spatial-temporal fusion module to balance the information obtained from other frames while achieving feature refinement. Experimental results on benchmark datasets show that the proposed NLBF is able to achieve state-of-the-art performance in the VSR task.
A Two-Stage 3D Object Detection Algorithm Based on Deep Learning

Object detection in point clouds is an important fundamental problem for many applications such as autonomous driving. The cutting-edge methods generally apply an end-to-end frame to detect the objects from the point clouds, while the traditional methods generally apply a multistep frame that clusters the points before detecting it. The end-to-end frame has been proven to be much better in many aspects, but the good detection effect is supported by a large amount of training data. As is well known, the difficulty of creating point clouds datasets is much greater than that of images datasets. Moreover, when the scenes undergo significant changes, the detection effect of end-to-end frame often decreases significantly. Therefore, on the basis of traditional detection algorithms, we propose a flexible and highly transferable two-stage algorithm AF3D where the point clouds is firstly clustered into clusters which were then detected by a classification network based on deep learning. We verify the algorithm on both the KITTI dataset and our own dataset, and compared it with Pointpillars. The conclusion indicates that AF3D has better robustness than Pointpillars in unfamiliar scenes.
When Diffusion Model Meets with Adversarial Attack: Generating Transferable Adversarial Examples on Face Recognition

Face recognition has witnessed continuous development along with advanced deep convolutional neural networks (DCNNs). However, DCNNs are vulnerable to adversarial examples, resulting in potential security risks to face recognition applications. While the adversarial examples generated by the gradient-based approaches are overfitting into the substitute model, sacrificing transferability. Therefore, this paper proposes a novel transfer-based black-box adversarial attack formulation with the diffusion model (DiffAttack). In particular, DiffAttack employs the dynamic noise from the diffusion process to increase the adversarial perturbation diversity. Besides, DiffAttack obtains ensemble-like effects by re-sampling tactics during the proceeding of diffusion. Accordingly, DiffAttack effectively prevents the overfitting issue, delivering more powerful model capacity. Extensive experiments are conducted on two benchmarking datasets, LFW and CelebA-HQ. The results demonstrate the effectiveness of DiffAttack, which achieves a higher attack success rate than the state-of-the-art, with comparable image quality to that of other transfer-based methods. Codes will be available on the project homepage: https://github.com/LiYuanBoJNU/DiffAttack .
Class Incremental Learning with Important and Diverse Memory

Class incremental learning (CIL) has been attracting increasing attention in computer vision and machine learning communities, where a well-known issue is catastrophic forgetting. To mitigate this issue, a popular approach is to utilize the replay-based strategy, which stores a small portion of past data and replays it when learning new tasks. However, selecting valuable samples from previous classes for replaying remains an open problem in class incremental learning. In this paper, we propose a novel sample selection strategy aimed at maintaining effective samples from old classes to address the catastrophic forgetting issue. Specifically, we employ the influence function to evaluate the impact of each sample on model performance, and then select important samples for replay. However, given the potential redundancy among selected samples when only considering importance, we also develop a diversity strategy to select not only important but also diverse samples from old classes. We conduct extensive empirical validations on the CIFAR10 and CIFAR100 datasets and the results demonstrate that our proposed method outperforms the baselines, effectively alleviating the catastrophic forgetting issue in class incremental learning.
TPM: Two-Stage Prediction Mechanism for Universal Adversarial Patch Defense

Deep neural networks can be deceived by universal adversarial patches, which introduce adversarial noise within a local region. Universal adversarial patches can be effectively used in diverse conditions, which leads to exposing the vulnerability of computer vision systems and raise a significant safety concern. To solve this problem, we propose a Two-stage Prediction Mechanism (TPM) for universal adversarial patch defense, which consists of two main operations: (1) The Region Treatment locates the most highlighted region via a global adaptive threshold and substitutes it with a mask, thus yielding a masked image that can effectively eliminate the potential universal adversarial patch. (2) The Category Prediction first synthesizes alternative contents from the masked pixels by image inpainting, thus yielding an inpainted image. Based on the above, the final output category is predicted via the weighted combination of the output logits between the masked and inpainted images. Extensive experiments on two datasets demonstrate the effectiveness of the proposed TPM in defending against universal adversarial patches and further show superior performance on multi-shape attacks, adaptive and multi-target attacks.
Audio-Visual Saliency for Omnidirectional Videos

Visual saliency prediction for omnidirectional videos (ODVs) has shown great significance and necessity for omnidirectional videos to help ODV coding, ODV transmission, ODV rendering, etc.. However, most studies only consider visual information for ODV saliency prediction while audio is rarely considered despite its significant influence on the viewing behavior of ODV. This is mainly due to the lack of large-scale audio-visual ODV datasets and corresponding analysis. Thus, in this paper, we first establish the largest audio-visual saliency dataset for omnidirectional videos (AVS-ODV), which comprises the omnidirectional videos, audios, and corresponding captured eye-tracking data for three video sound modalities including mute, mono, and ambisonics. Then we analyze the visual attention behavior of the observers under various omnidirectional audio modalities and visual scenes based on the AVS-ODV dataset. Furthermore, we compare the performance of several state-of-the-art saliency prediction models on the AVS-ODV dataset and construct a new benchmark. Our AVS-ODV datasets and the benchmark will be released to facilitate future research.
Cross-Modality Fused Graph Convolutional Network for Image-Text Sentiment Analysis

Image-text sentiment analysis aims to mine the emotional semantics conveyed by image and text information. Psychological research shows that the conveyed sentiment depends on the long-range correlation and interaction of sentiment contextual features within and between the modalities of text and image. However, most of the previous works use the feature stitching strategy based on gating and attention mechanism to fuse image-text data, and do not fully exploit the relationship within and between modalities. In this paper, we propose a cross-modality fused graph convolutional network for image-text sentiment classification. The proposed network reasons on intra-modal and inter-modal graphs to realizes the fusion between text and image. In addition, the huge intra class variability of multimodal data affects the accuracy of sentiment analysis. Therefore, we design a sample sentiment similarity loss function to help the model learn sentiment-related similar features from image and text data. Extensive experiments on image-text sentiment datasets show that the proposed network outperforms the existing sentiment analysis models.
Distortion-Aware Mutual Constraint for Screen Content Image Quality Assessment

The relationship between image distortion and image quality exhibits a strong correlation. Various methods incorporate auxiliary tasks such as the classification of distortion types and levels to address this issue. However, these tasks tend to be relatively independent and do not exert significant mutual influence. To tackle this challenge, we propose a novel method called the distortion-aware mutual constraint (DAMC) for full-reference screen content image quality assessment. Our approach consists of a feature extraction network and a multi-task network, with the DAMC loss function facilitating positive feedback and entanglement among all tasks to achieve simultaneous optimization. To transform the origin image features space for the following main and auxiliary tasks, we propose a selective similarity fusion (SSF) module which weights different similarities in accordance with the origin image features. The experimental results demonstrate the excellent performance and generalization capability of our method.
Implicit Representation for Interacting Hands Reconstruction from Monocular Color Images

Reconstructing interacting hands from monocular color images plays an important role in promoting the understanding of human behavior and the application of existing AR/VR technology. With the emergence of interacting hands datasets, numerous data-driven methods attempt to regress the MANO parameters of the left and right hand from the network. This formulation not only limits the handedness and the number of hands in the image but also introduces the deficiencies of the MANO into the interacting-hand reconstruction task. To address this obstacle, we present a novel framework for representing the surface of interacting hands with implicit functions. Our key idea is to construct an implicit neural occupancy network, which implicitly reconstructs interacting hands with image features as query strategies and conditions. The proposed query strategies only require one sample to obtain enough points on the surface. Our unified framework efficiently completes the task of reconstructing hands with the unlimited number, handedness, and interaction compactness from monocular color images.
Cyber and Digital Forensic

Digital forensic includes a wide variety of spectrum, which deals with the mobile forensic, computer forensic, network forensic or other types of different devices related to the digital evidence. The main goal of digital forensic is to focus on the recovery and investigation of artifacts found in digital evidences related to cyber-crimes. In comparison to various other domains of forensics digital forensics has evolved faster as simple because technology changes rapidly. The advancement and upgradation of forensic software’s are needed with each new version of an operating system, digital forensic must keep up with advances in technology. It is withal important to maintain the chain of custody from proper collection, seizing and transportation. In the forensic laboratories there are various forensically sound software which are used for examination in which extraction and interpretation is done. This needs a well sound scientist having a great knowledge. Also, there are different types of certifications available by many of the renowned organizations.
U-TEN: An Unsupervised Two-Branch Enhancement Network for Object Detection Under Complex-Light Condition

The goal of low-light enhancement is to improve the visual quality of dark regions in an image. However, the existing low-light enhancement methods always failed in nighttime traffic surveillance. The reason comes from complex-light instead of low-light, while traditional methods can hardly treat regions with different light conditions. In this paper, in order to solve above challenge, we propose an unsupervised tow-branch complex-light enhancement network (U-TEN) with graph attention network and generative adversarial network to enhance pixels in different regions according to their light conditions. To solve the problem that enhance weights are supposed to varied with regions, we proposed two-branch attention block. It aims to model two kinds of light information interaction. U-TEN can improve the object detection performance under complex-light condition in nighttime traffic surveillance. The proposed U-TEN is verified on ODLS dataset, and the results show that our network improves the object detection average precision by 3.50% on average, and 4.93% in maximum. These results demonstrate the proposed U-TEN has great potential to be used in vision applications of nighttime traffic surveillance.
Research on Construction of Higher Vocational Teaching Quality Evaluation System Based on BP Neural Network

With the development of education today, how to improve the teaching of higher education is an important link in the education cycle. From the experience of higher vocational education in recent years, teaching effect will be affected by teaching environment, teachers’ ability, students’ talent, and many other factors. Based on the understanding of the status quo of higher education and the basic concept of BP neural network, this paper studies how to use BP neural network to create evaluation and test the effectiveness of BP neural network in higher education. The experimental results show that BP neural network plays an important role in higher level training evaluation, and the overall evaluation is feasible.
Skeleton Based Dynamic Hand Gesture Recognition using Short Term Sampling Neural Networks (STSNN)

This research introduces an innovative framework for real-time dynamic hand gesture recognition in the field of Human-Computer Interaction (HCI). The framework combines depth learning networks with the integration of multiple datasets to extract both short-term and long-term features from video input. A significant contribution of this research lies in the integration of Convolutional Neural Networks (CNNs) into a specialized short-term memory network (STSNN), enabling the capture of long-term contextual information for accurate gesture recognition. The proposed framework is thoroughly evaluated using two hand-held databases, namely the 14/28 dataset and the LDMI database. By leveraging the computational power of depth learning networks and the fusion of diverse datasets, our model outperforms previous methods, establishing its efficacy in real-time dynamic hand gesture recognition tasks. The outcomes of this research significantly contribute to the advancement of HCI, providing a robust and technically sophisticated solution for gesture-based interfaces. The findings hold promise for enhancing user experiences and facilitating seamless integration of gesture-based interaction techniques across various domains, ultimately improving the efficiency and effectiveness of human-computer interactions.
FAFormer: Foggy Scene Semantic Segmentation by Fog-Invariant Auxiliary Domain Adaptation

Semantic Segmentation in the Foggy Scenes (SSFS) remains a difficult problem due to uncertainties caused by imperfect observations. Considering the success of domain adaptive semantic segmentation in the clear scenes, we believe it is reasonable to transfer the knowledge from the clear images to the foggy images. Different from the previous methods which mainly focus on alignment between the clear domain and the foggy domain, we try to transfer the fog knowledge between different domains to a “teacher” segmentor, thus the latter can generate better pseudo labels to supervise the student segmentor (main segmentor) to close the domain gap. Our method achieved better performance on ACDC and Foggy Zurich benchmark compared with mainstream works.