Reviewerly/exHarmony huggingface dataset:
Semi-fragile neural network watermarking for content authentication and tampering localization

As an emerging digital product, artificial intelligence models face the risk of being modified. Malicious tampering will severely damage model functions, which is different from normal modifications. In addition, tampering localization for targeted repair can effectively reduce the cost. Therefore, it is crucial to achieve model content authentication and locate the tampering location. We proposed a novel semi-fragile neural network watermarking method in this paper to address these issues. Specifically, with the precondition of maintaining model performance, we proposed a method that generates a set of semi-fragile samples for a model to achieve the content authentication and tampering localization. The experiment results show that the content authentication of the model can be achieved by analyzing the output results of the model for the semi-fragile samples. When the model is processed normally, the output results are consistent with the expected label, while when the model is maliciously tampered with, the model produces unstable output. Furthermore, the tamper localization of the model can be further achieved through the information hidden in the semi-fragile samples, resulting in an average accuracy of more than 99.42%. In addition, our method is also effective for other deep neural networks.
A Genetic Algorithm-based sequential instance selection framework for ensemble learning

The accumulation of large amounts of historical data has led to the wide application of ensemble learning over the past few decades, but the balance between the individual accuracy of base classifiers (BCs) and the diversity among these BCs is rarely considered in the construction of ensemble models. Since such a balance is crucial to the success of ensemble models, this paper proposes a Genetic Algorithm-based sequential instance selection framework to address this research gap. The novelties of the proposed framework include: transforming the balance between the individual accuracy of BCs and the diversity among BCs into a general combinatorial optimization model and designing a Genetic Algorithm-based evolutionary instance selection method to solve this model. The proposed framework not only overcomes the inherent limitations of the Genetic Algorithm in some high-dimensional tasks but also provides an explicit and automatic way to balance the accuracy and diversity by searching appropriate training data subsets for different component BCs. Based on obtained training data subsets, the component BCs of the ensemble model are generated sequentially, and their predictions are further combined with the weighted majority voting rule. Using 30 real datasets collected from various practical applications, such as medicine, business, and industry, the effectiveness of the proposed framework in constructing powerful ensemble models is examined and compared with six benchmark ensemble learning methods. In addition, the capability of the proposed framework to improve convergence performances is also examined by the comparison with the traditional Genetic Algorithm.
Eye detection and coarse localization of pupil for video-based eye tracking systems

A video-based eye tracking system generally captures NIR images, each of which contains one or two eyes of a subject. The subject's point of gaze is then determined using 3D eye model and pupil centre corneal reflection technique. Eye detection and pupil localization play significant roles in video-based eye tracking systems. However, face rotation, wearing glasses, eye-shape variation and illumination variation make it difficult to detect an eye and localize a pupil accurately in the images captured by video-based eye tracking systems. In this paper, we proposed an eye detector that adopts a coarse-to-fine strategy. The eye detector consists of three classifiers: an ATLBP-THACs feature-based cascade classifier, a branch CNN and a multi-task CNN. We also proposed a method for coarse pupil localization. Coarse localization is an important step for pupil localization since it can provide initial pupil coordinates for a fine pupil localization method. Given a downscaled eye image, a shallow CNN is used to estimate the location of seven landmarks. On this basis, pupil center and radius are estimated. A method for small dim target enhancement is used to increase the contrast between pupil and background. The main goal of pupil enhancement is to make it easier to binarize an eye image. At last, component filtering is made by utilizing the estimated pupil center and radius. We collected a dataset named neepuEYE dataset that consists of 5500 NIR eye images from 109 people. The images can be used to generate augmented samples for training an eye detector since they contain eyes with different shape, orientation and pupil localization. Experimental results show that our eye detector is a fast and robust detector. Furthermore, our method for coarse pupil localization can obtain not only high detection rate but also high localization speed.
Deep multi-view graph clustering network with weighting mechanism and collaborative training

With the development of graph convolutional network (GCN), which is powerful in graph embedding learning meanwhile can capture node feature information, deep multi-view graph clustering methods based on graph autoencoder have emerged as a new stream. Although they achieve satisfactory performance, they still have the following weaknesses: (1) some of them model the weights for different views in the encoder part by using attention in the multi-view embedding fusion layer, but fail to consider the weighting in the decoder part to measure the contributions of different views to the reconstruction loss. (2) Most of them directly conduct clustering on the multi-view common embedding layer, but fail to guarantee the alignment of clustering results of different views. To this end, we propose a novel GCN-based deep multi-view graph clustering network with weighting mechanism and collaborative training (DMVGC). The model is composed of multiple view-specific graph encoders and a unified graph decoder. Besides the specially-designed attention module in the encoder part, we construct a reconstruction loss with adaptive weighting mechanism in the decoder part. Additionally, a collaborative self-training clustering objective is jointly conducted on each view-specific embedding layer and the common embedding layer, to make the embedding of each view clustering-friendly toward a common partition. Experiments on several datasets demonstrate the effectiveness of our model.
Multi-site solar irradiance forecasting based on adaptive spatiotemporal graph convolutional network

Accurate solar irradiance forecasting with fine spatiotemporal correlations is essential for photovoltaic power generation. However, current spatiotemporal methods are unable to extract spatial features appropriately, leading to rough prediction results. Consequently, a deep graph neural network is proposed in this paper for multi-site solar irradiance forecasting. Self-adaptive adjacency matrices were developed to better capture the intrinsic influence between sites under different climate types at various time steps. Based on the adaptive graph convolution and long short-term memory network (AGCLSTM), the spatiotemporal features of each site were extracted to obtain prediction results for all the sites in the research area simultaneously. In the experiments conducted on the datasets of three climate types, the average root mean square error (RMSE), mean absolute error (MAE), and correlation coefficient (R) of the proposed model are observed to be in a range of 47.519–54.854 W/m2, 18.968–24.426 W/m2, and 0.9737–0.9889, respectively. Compared to the benchmark models, the average RMSE and MAE reduce by 1.31%–10.86% and 3.75%–18.57%, respectively. Moreover, the proposed method proves to be effective for scenes of solar irradiance fluctuations with the help of self-adaptive adjacency matrices, which enhances its generalization ability and prediction accuracy for one-hour-ahead solar irradiance forecasting.
Advanced strategies on update mechanism of tree-seed algorithm for function optimization and engineering design problems

Tree-seed algorithm (TSA) is one of the meta-heuristic algorithms proposed recently. It has a simple but effective structure. However, for complex high-dimensional functions and constrained optimization problems, TSA has the defects of slow convergence rate and low optimization accuracy. To overcome these issues, an improved tree-seed algorithm (SATSA) based on Nelder-Mead simplex method is proposed. Firstly, appropriately increase the value of the control parameters that balance exploration and exploitation, thereby improving the local search ability of TSA, and introduce an adaptive stochastic component of Gaussian distribution in the seed generation formula of the local search stage. Then, set the maximum retention times of each tree, for trees that exceed the maximum retention times in continuous iterations, increase the number of seeds produced by this tree, then use a new equation based on the global optimal location to generate seeds, thereby speeding up the convergence of the algorithm. Finally, after all trees produce seeds, introduce the improvement strategy with Nelder-Mead simplex method into some poor quality solutions, so that the solution with poor quality is searched deeply in the neighborhood, speeding up the process of moving the overall solution to the optimal solution. Its convergence is proved by theoretical analysis, and certifies that the time complexity of SATSA is not increased compared to TSA. The simulation results of 6 advanced algorithms on CEC2017 test suite and 10 challenging engineering constrained optimization problems demonstrate that the three strategies proposed in this study are feasibility and effectiveness.
Class imbalance mitigation: A select-then-extract learning framework for emotion-cause pair extraction

The class imbalance problem arises in the current end-to-end models for the emotion-cause pair extraction (ECPE) task, and a large number of negative clause pairs interfere model to depict the characteristics of emotion and cause. Most existing methods adopt a window-constrained strategy to limit the relative distance between candidate clauses to a smaller range to filter negative pairs. However, those methods ignore the semantic association between distant clauses, and using an artificial window size is imperfect due to the inherent ambiguity and delicacy of emotion. Inspired by the centrality principle of discourse, in this paper we propose a select-then-extract framework for the ECPE task, where the core clauses are first selected from the original document as candidate emotion and cause clauses, and then emotions, causes and emotion-cause pairs are jointly extracted from the candidate clauses. Specifically, our model includes two main components: core clause selector and emotion-cause pairs extractor. For the Core Clause Selector, we introduce the extractive document summarization (EDS) task and present a multi-granularity semantic awareness cooperative graph model (MGCOG) to extract core clauses from documents. Compared to the previous methods, the core clause selector is more effective for alleviating the category imbalance because the global causal cues and context can be captured by learning global keywords and inter-clause relationships, which determines more efficient candidate clauses. For Emotion-Cause Pairs Extractor, we put forward a multi-task learning model to jointly extract emotions, causes and emotion-cause pairs from the selected core clauses. Here, a multi-head attention is used to further model the relationship between candidate clauses, and a co-predictor is designed for assigning scores to all possible emotion-cause pairs. We further investigate the pipeline and the joint model under the select-then-extract framework, and show that the experimental results on benchmark datasets are consistently superior to the comparative baseline models. Extensive ablation experiments also verify the effectiveness of each component.
Realistic feature perception for face frontalization with dual-mode face transformation

Despite face frontalization developing rapidly, limited image pairs and low image quality are still obstacles to face frontalization. Although single-view-based face frontalization does not have the constraints of image pairs, it reduces the exploration of unknown data. We alleviate the face distortion problem based on single-view face frontalization from two aspects. One is that we designed a dual-mode face transformation model for model training based on the loss of facial features from different angles. These two training modes cope with the question of face frontalization in different distortions. Specifically, we apply the 3D head model to the data transformation by fitting the 2D image to the 3D model. Rotating the 3D model at the designed angle to realize the face transformation. We judge the distortion extent with Euler Angle and produce the two training modes corresponding to the dual-mode face transformation. On the other hand, we construct a perceptual loss module to preserve the detailed information of the front view. This module combines content and identity loss to generate the photorealistic frontal image. In qualitative and quantitative experiments, we conduct experiments to illustrate the advantages of the proposed model on controlled and field datasets. Further, we apply full-reference and no-reference image quality assessment methods to estimate the quality of images generated by different face frontalization models.
Traffic Sign Recognition Using Optimized Federated Learning in Internet of Vehicles

Traffic Sign Recognition (TSR) is vital for vehicle safety and navigation, especially in the era of autonomous cars. Internet of Vehicles (IoV) provide a promising infrastructure for vehicular networks due to their agility and interoperability. However, privacy concerns and network restrictions hinder the collection of massive data from distributed automotive sensors in IoV. To address these challenges, this paper proposes the application of Federated Learning (FL) and model sparsification to optimize TSR in autonomous vehicles. FL enables decentralized learning while preserving data privacy, and model sparsification significantly reduces communication costs. Furthermore, we incorporate the Adam optimizer for local training, ensuring efficient model optimization on each vehicle. Experimental results demonstrate the effectiveness of our approach, with improved TSR performance while mitigating privacy risks and enhancing communication efficiency. This research contributes to the advancement of TSR in IoV by introducing FL, model sparsification, and the use of the Adam optimizer for local training, facilitating efficient and privacy-preserving vehicular network learning.
Enhancing texture representation with deep tracing pattern encoding

Texture representation is a challenging problem due to the complex underlying physics of texture as well as the variations caused by changes in viewpoint. Recent progress in texture analysis has been made by the power of convolutional neural networks (CNNs) in feature learning. However, most current methods aggregate the features from the last convolutional layer of the CNN to obtain a global feature vector, which fails to leverage shallow low-level visual cues and cross-layer feature patterns, limiting their performance. In this paper, we propose to trace the features generated along the convolutional layers via a histogram of local 3D invariant binary patterns, called deep tracing patterns. This leads to a highly discriminative yet robust global feature representation module. Building such a module into a CNN backbone, we develop an effective approach for texture recognition. Extensive experiments on six benchmark datasets show that the proposed approach provides a discriminative and robust texture descriptor, with state-of-the-art performance achieved.
Performance evaluation for multi-target tracking with temporal dimension specifics

With the great development of Multi-Target Tracking (MTT) technologies, many MTT algorithms have been proposed with their own advantages and disadvantages. Due to the fact that requirements to MTT algorithms vary from the application scenarios, performance evaluation is significant to select an appropriate MTT algorithm for the specific application scenario. In this paper, we propose a performance evaluation method on the sets of trajectories with temporal dimension specifics to compare the estimated trajectories with the true trajectories. The proposed method evaluates the estimate results of an MTT algorithm in terms of tracking accuracy, continuity and clarity. Furthermore, its computation is based on a multi-dimensional assignment problem, which is formulated as a computable form using linear programming. To enhance the influence of recent estimated states of the trajectories in the evaluation, an attention function is used to reweight the trajectory errors at different time steps. Finally, simulation results show that the proposed performance evaluation method is able to evaluate many aspects of the MTT algorithms. These evaluations are worthy for selecting suitable MTT algorithms in different application scenarios.
Characteristics of students’ learning behavior preferences — an analysis of self-commentary data based on the LDA model

How to better grasp students’ learning preferences in the environment of rapid development of engineering and science and technology so as to guide them to high-quality learning is one of the important research topics in the field of educational technology research today. In order to achieve this goal, this paper utilizes the LDA (Latent Dirichlet Allocation) model for text mining of the survey results on the basis of a survey on students’ self-perception evaluation. The results show that the LDA model is capable of extracting terms from text, fuzzy identifying groups of students at different levels and presenting potential logical relationships between the groups, and further analyzing the learning preferences of students at different levels for IT courses. Based on the student’s learning needs, this paper proposes recommendations for developing students’ learning effectiveness. The LDA method proposed in this paper is a feasible and effective method for assessing students’ learning dynamics as it generates cognitive content about students’ learning and allows for the timely discovery of students’ learning expectations and cutting-edge dynamics.
Shadow-aware dynamic convolution for shadow removal

With a wide range of shadows in many collected images, shadow removal has aroused increasing attention since uncontaminated images are of vital importance for many computer vision tasks. Current methods consider the same convolution operations for both shadow and non-shadow regions while ignoring the large gap between the color mappings for the shadow region and the non-shadow region, leading to poor quality of reconstructed images and a heavy computation burden. To solve this problem, this paper introduces a novel plug-and-play Shadow-Aware Dynamic Convolution (SADC) module to decouple the interdependence between the shadow region and the non-shadow region. Inspired by the fact that the color mapping of the non-shadow region is easier to learn, our SADC processes the non-shadow region with a lightweight convolution module in a computationally cheap manner and recovers the shadow region with a more complicated convolution module to ensure the quality of image reconstruction. Given that the non-shadow region often contains more background color information, we further develop a novel intra-convolution distillation loss to strengthen the information flow from the non-shadow region to the shadow region. Extensive experiments on the ISTD and SRD datasets show our method achieves better performance in shadow removal over many state-of-the-art methods. Codes have been made available at https://github.com/xuyimin0926/SADC.
Image inpainting exploiting global prior refined weighted low-rank representation

Low-rank and nonlocal self-similarity are important priors for image restoration. Based on nonlocal self-similarity assumption, similar patches are grouped into a matrix that is of low-rank or approximately low-rank. Traditional algorithms model low-rank prior of local group matrices (intra-group) and ignore the global inter-group relationship, which will result in suboptimal performance without considering global consistency. Due to overlapping of patches, groups are dependent, however, conventional methods process each group independently. To make full use of the intrinsic relationship among groups, we consider all groups simultaneously and propose a global prior refined weighted low-rank representation model for image inpainting. The local-groups and the aggregation of all groups are taken into account in the proposed model. By introducing auxiliary variable, we decouple the local–global model and optimize iteratively using the alternating direction method of multipliers. The proposed global refinement model significantly improves the performance of local low-rank model. A new strategy for grouping similar patches is also proposed by dividing the neighborhood of a target patch into multiple subregions and searching for a given number of similar patches within each subregion. The proposed method is applied to various tasks (line inpainting, destriping, and depth map completion) and obtains promising performance.
Underwater image enhancement utilizing adaptive color correction and model conversion for dehazing

Underwater imaging is typically affected by absorption and scattering, which can lead to color distortion and image blurring. However, no universal color constancy algorithm for correcting underwater images exists due to the wide variation in underwater scenes. Furthermore, methods based on learning to improve underwater image quality often suffer from a lack of ground truth, which can compromise the validity and authenticity of restored images. In this study, we propose an adaptive underwater image enhancement technique that corrects image color and reduces haze using a valid dataset with ground truth. Specifically, we first use hue channel statistics from underwater images to build a dataset of color-corrected images from different underwater scenes. We use this dataset to train an Unet-like network for adaptive color correction. After color correction, underwater images often exhibit characteristics similar to hazy terrestrial images. We refer to this phenomenon as “model conversion”. The hazy terrestrial images have corresponding ground truth. Therefore, we train a Transformer-like network with hazy terrestrial image datasets to remove haze in underwater images. Our method is more robust for different underwater scenes and elegantly solves the problem of lacking ground truth. A series of experiments demonstrate that our method achieves superior performance compared to state-of-the-art methods in terms of both visual quality and quantitative metrics.