Reviewerly/exHarmony huggingface dataset:
Feature Representation Enhancing by Context Sensitive Information in CTR Prediction

Click-Through-Rate (CTR) is a fundamental metric used to assess the efficacy of recommendation systems. In the past, most CTR prediction approaches focused on modeling the cross feature of various feature fields to improve the accuracy of CTR prediction. But they only learned the fixed representation of feature and neglected the varying significance of different feature fields in distinct contexts - what we refer to as context sensitive information - leading to suboptimal performance. While recent approaches have attempted to leverage linear transformations and feature interactions to capture context sensitive information, they remain inadequate as they overlook the varying importance of original features or different order cross features. In this paper, we propose a new module called Enhancing Feature Network (EFNet). EFNet has two key components: 1) Information Capture Layer (ICL), that dynamically captures explicit and implicit context sensitive information from original embedding features and digs out their corresponding bit-level weights; 2) Enhancing Feature Layer (EFL) that adaptively combine the context sensitive information with original embedding features according to the weights obtained in ICL. It is worth noting that EFNet can be integrated into existing CTR prediction models as a module to boost their overall performance. We conduct comprehensive experiments on four public datasets and the results demonstrate that models incorporating the EFNet module outperform other state-of-the-art models.
Discovering Skyline Periodic Itemset Patterns in Transaction Sequences

As an extended version of frequent itemset patterns, periodic itemset patterns concern both the frequency and periodicity of itemsets at the same time, so they contain more information than frequent itemset patterns, which only concern the frequency. With further research, we found that, in some cases, the periodic itemset patterns with higher frequency, or with optimal periodicity, or with both higher frequency and optimal periodicity have higher application value. However, there is currently no work focusing on such a kind of periodic itemset patterns. In view of this, this paper first proposes a new concept of skyline periodic itemset patterns, and states the problem of skyline periodic itemset pattern mining, then presents an algorithm called SLPIM (SkyLine Periodic Itemset pattern Miner) for skyline periodic itemset pattern mining. SLPIM first adopts the well-known FP-Growth algorithm to mine all frequent itemset patterns, and then uses an effective judgment strategy to determine which frequent itemset patterns are skyline periodic itemset patterns. Finally, experiments are conducted on two real-world and two simulated datasets. The results show that SLPIM is competent for mining skyline periodic itemset patterns.
AAP: Defending Against Website Fingerprinting Through Burst Obfuscation

Website fingerprinting enables eavesdroppers to identify the website a user is visiting by network surveillance, even if the traffic is protected by anonymous communication technologies such as Tor. To defend against website fingerprinting attacks, Tor provides a circuit padding framework as the official way to implement padding defenses. However, the circuit padding framework can not support additional delay, which makes most defense schemes unworkable. In this paper, we study the patterns of HTTP requests and responses generated during website loading and analyze how these high-level features correlate with the underlying features of network traffic. We find that the HTTP requests sent and responses received continuously in a short period of time, which we call HTTP burst, have a significant impact on network traffic. Then we propose a novel website fingerprinting defense algorithm, Advanced Adaptive Padding(AAP). The design principle of AAP is similar to Adaptive Padding, which works by obfuscating burst features. AAP does not delay application packets and is in line with the design philosophy of low latency networks such as Tor. Besides, AAP uses a more sensible traffic obfuscation strategy, which makes it more effective. Experiments show that AAP outperforms other zero-delay defenses with moderate bandwidth overhead.
Multi-Self-Supervised Light Graph Convolution Network for Social Recommendation

Self-supervised learning is a novel paradigm between unsupervised and supervised learning, which can use the ground-truth samples automatically generated from the raw data for self-identification based contrastive learning and effectively alleviate the problem of data sparsity. Its ingenious combination with graph convolution network is a major breakthrough in the current recommendation field. However, mostly existing methods fail to fully consider the interference factors in social relations, fail to effectively improve the deviation of predicting scores only relying on the user and item representations, and fail to take into account the time cost of model training while improving the recommendation performance. To address these issues, this paper proposes a social recommendation framework based on multi-self-supervised light graph convolution network, named as MSR. Technically, MSR first utilizes the user-item sampling interaction graph to enhance the learning of user/item interaction representations, and then further enhances the learning of user social representations by fusing social relations through two-stage encoding. Finally, the interaction representations, social representations, and item bias are combined to predict scores. Our experiments on two real-world datasets LastFM and Douban-Book demonstrate that, compared with the current mainstream models, MSR improves the Precision@10 at least 2.35% and 10.41%, respectively, and reduces the time consumption at least 35.35% and 76.48%, respectively.
A Hessian-Based Federated Learning Approach to Tackle Statistical Heterogeneity

Federated learning (FL) involves collaboration between clients with limited data to produce a single optimal global model through consensus. One of the difficulties with FL is the differences in data statistics between local clients. Clients with statistically heterogeneous data deviate from the global target, resulting in a slower convergence rate and increased communication resource consumption. To address this problem, we propose a new approach, FedH, that maintains the proximity of local models to the global target while maximizing communication efficiency and computational resources. We use the Hessian matrix to constrain client updates that deviate from the global target. Our results demonstrate the superiority of FedH over FL baselines such as FedAvg, FedProx, and Fedcurv when applied to benchmark datasets such as MNIST, Fashion-MNIST, and CIFAR-10 across a range of statistical heterogeneity levels.
A Novel Variational Autoencoder with Multi-position Latent Self-attention and Actor-Critic for Recommendation

Variational Autoencoder (VAE) has been extended as a representative nonlinear latent method for collaborative filtering recommendation. As a high-dimensional representation of data, latent vectors play a vital role in the transmission of important information in a VAE model. However, VAE-based models suffer from a common limitation that the transmission ability of the latent vectors’ important information is limited, resulting in lower quality of global information representation. To address this, we present a novel VAE model with multi-position latent self-attention and reinforcement learning’ actor-critic algorithm. We first build a multi-position latent self-attention model, which can learn richer and more complex latent vectors and strengthens the transmission of important information at different positions. At the same time, we use reinforcement learning to enhance the interactive learning process of collaborative filtering recommendation training. Specifically, our model is stable and can be easy applied in the recommendation. We observed significant improvements over the previous state-of-the-art baselines on three social media datasets, where the largest improvement can reach $$26.10\%$$ .
Incremental Natural Gradient Boosting for Probabilistic Regression

The natural gradient boosting method for probabilistic regression $$(\mathrm {{\textbf {NGBoost}}})$$ is capable of predicting not only point estimates but also target distributions under sample conditions, thereby quantifying prediction uncertainty. However, NGBoost is designed only for batch settings, which are not well-suited for data stream learning. In this paper, we present an incremental natural gradient boosting method for probabilistic regression $$(\mathrm {{\textbf {INGBoost}}})$$ . The proposed method employs scoring rule reduction as a metric and applies the Hoeffding inequality incrementally to construct decision trees that fit the natural gradient, thus achieving incremental natural gradient boosting. Experimental results demonstrate that INGBoost performs well in both point regression and probabilistic regression tasks while maintaining the interpretability of the tree model. Furthermore, the model size of INGBoost is significantly smaller than that of NGBoost.
SSM: Semantic Selection and Multi-view Alignment for Image-Text Retrieval

Image-text retrieval has been a crucial and fundamental task in multi-modal field. Benefiting from the superiority of Transformer encoder in modeling multimodal information, the Transformer-based alignment model has become the mainstream of image-text retrieval. However, current Transformer-based alignment models suffer from two major limitations: (1) The redundancy of modal features and the complexity of correlations between modalities restrict the performance of the model. (2) Current researches are typically limited to a single viewpoint during the modal alignment. To address these issues, in this paper we propose a image-text retrieval model SSM based on Semantic Selection and Multi-view alignment. Specifically, we introduce a gated attention unit to filter unnecessary information, and design an adaptive weighted similarity calculation method to dynamically adjust the importance of different features during the alignment process. On the other hand, we design a multi-view cross-modal alignment method that considers different granularity and different level of information to provide complementary benefits in representation learning. We compare SSM with other advanced image-text retrieval models in MS-COCO and Flickr30K datasets, and the results show that the SSM model has competitive performance without much interaction.
Aspect-Based Sentiment Analysis via BERT and Multi-scale CBAM

Aspect-based sentiment analysis (ABSA), as a fine-grained sentiment analysis task, predicts polarities for given aspects in one text. However, in the process of aspect-based sentiment analysis, there may be multiple perspectives of sentiment-prone text information, as well as implicit sentiment expressions, which can cause greatly influence on the accuracy of sentiment analysis results. To address the above problems, in this paper, we propose an aspect-based sentiment analysis model BERT-MSCBAM based on BERT and Multi-Scale Convolutional Block Attention Module (MSCBAM). Our model firstly uses BERT to encode the input text, then uses the MSCBAM module, which extracts the deep semantic and key features by its structure of two Multi-Scale Channel Attention Modules (CAMs) with different scales and one Spatial Attention Module (SAM) between them, combining with ResNet, and finally obtains the prediction results through a fully connected layer. To validate the effectiveness of our BERT-MSCBAM model, we conduct experiments on the Restaurant and Laptop datasets of SemEval2014 Task 4, and the Twitter dataset for our model in comparison with several current mainstream models. The experimental results validate the effectiveness of our model.
Knowledge-Rich Influence Propagation Recommendation Algorithm Based on Graph Attention Networks

One of the biggest challenges in Recommendation Algorithms (RA) is how to obtain user and item embeddings from sparse interaction history. To take this challenge, most graph neural network based RAs explicitly incorporate high-order collaborative filtering signals on the user-item bipartite graph with either multi-layer semantics on the Knowledge Graph (KG) or multi-level neighbors on the social network. However, none of them fully integrate these three types of graph-structured data, which decreases embeddings’ precision. Based on this consideration, this paper integrates the three types of data by proposing a knowledge-rich influence propagation RA based on the graph attention mechanism. Specifically, in the semantic propagation, we categorize user preferences into deep interest obtained by multiple graph attention message propagations on related KG parts, and shallow interest generated from the interaction history. Moreover, the influence weight between items is determined by the number of co-interactions and the semantic similarity. These two factors as well as social relations together decide the influence weight between users. With these influence weights, final user and item embeddings are calculated through multi-layer message propagation. The experimental results show that the proposed recommendation algorithm outperforms several compelling baselines on six scaled-down real-world datasets. This work has confirmed the effectiveness of combining these three types of data to increase RAs’ coverage and accuracy.
Training Noise Robust Deep Neural Networks with Self-supervised Learning

Training accurate deep neural networks (DNNs) on datasets with label noise is challenging for practical applications. The sample selection paradigm is a popular strategy that selects potentially clean data from noisy data for noise-robust training. In this study, we first analyze the sample selection models and find that the key aspects of this paradigm are the scale and degree of purity of the selected samples; however, these are restricted by the noise strength in the training set and the learning capacity of the models. Therefore, we propose a simple yet effective method called CoPL, which cross-trains two noise-robust DNNs simultaneously based on small loss criteria, along with learning accurate pseudo-labels. Benefitting from pseudo-labels, CoPL can reduce the noise strength and further promote the learning capacity and robustness of the models. Additionally, we discuss CoPL from the perspective of label smoothing to provide a theoretical guarantee of its performance. Extensive experimental results on both simulated (MNIST, CIFAR-10 and CIFAR-100) and real-world datasets (Clothing1M) demonstrate that CoPL is superior to other state-of-the-art methods and obtains a more noise-robust learning capacity.
Joint Modeling of Local and Global Semantics for Contrastive Entity Disambiguation

Entity disambiguation (ED) is a critical natural language processing (NLP) task that involves identifying and linking entity mentions in the text to their corresponding real-world entities in reference knowledge graphs (KGs). Most existing efforts perform ED by firstly learning the representations of mention and candidate entities using a variety of features and subsequently assessing the compatibility between mention and candidate entities as well as the coherence between entities. Despite advancements in the field, the limited textual descriptions of mentions and entities still lead to semantic ambiguity, resulting in sub-optimal performance for the entity disambiguation task. In this work, we propose a novel framework LogicED, which considers both Local and global semantics for contrastive Entity Disambiguation. Specifically, we design a local contextual module, which utilizes a candidate-aware self-attention (CASA) model and the contrastive learning strategy, to learn robust and discriminative contextual embeddings for both mentions and candidate entities. Furthermore, we propose a global semantic graph module that takes into account both the local mention-entity compatibility and the global entity-entity coherence to optimize the entity disambiguation from a global perspective. Extensive experiments on benchmark datasets demonstrate that our proposed framework surpasses the state-of-the-art baselines.
Iterative Encode-and-Decode Graph Neural Network

Graph neural networks (GNNs) have been extensively explored due to semi-supervised learning on graphs, which uses few labels to complete tasks without employing costly labeling information. Related methods are dedicated to mitigating the over-smoothing phenomenon to generate reliable node representations. However, existing methods lack correct guidance for neighbors and links from graph characteristics to node representations, resulting in incorrect neighbor information aggregation and poor representation discriminability. In this paper, we introduce a novel encoding and decoding framework that correctly leverages structure guided by labels and uses features for self-supervision of representations to alleviates the over-smoothing phenomenon, dubbed as Iterative Encode-and-Decode Graph Neural Network (IEDGNN). First, we offer a central component reconstruction module to correct the category centers of node representations, lowering the likelihood of aggregating neighbor information across categories. Then, we propose a feature self-reconstruction module that enables node representations to contain valid original attributes, making representations more informative in downstream classification tasks. We also theoretically analyze the impact of different encoder-decoder combinations on representation generation in our design. Extensive experiments demonstrate that our IEDGNN outperforms the state-of-the-art models on eight graph benchmark datasets with three label ratios.
Enhancing Multimedia Recommendation Through Item-Item Semantic Denoising and Global Preference Awareness

Multimedia recommendation aims to predict whether users will interact with multimodal items. A few recent works that explicitly learn the semantic structure between items using multimodal features manifest impressive performance gains. This is mainly attributed to the capability of graph convolutional networks (GCNs) to learn superior item representations by propagating and aggregating information from high-order neighbors on the semantic structure. However, they still suffer from two major challenges: a) the noisy relations (edges) in the item-item semantic structure disrupt information propagation and generate low-quality item representations, which impairs the effectiveness and robustness of existing methods; b) the lack of an optimization objective that exploits informative samples and global preference information leads to suboptimal training of the model, which makes users and items indistinguishable in the embedding space. To overcome these challenges, we propose Enhancing Multi media Recommendation through Item-Item Semantic Denoising and Global Preference Awareness (MMGPA). Specifically, the model contains the following two components: (1) a modal semantic representation network is carefully designed to learn the high-quality multimodal representation of items by modeling the denoised item-item semantic structure, and (2) a global preference-aware optimization objective prioritizes the most informative hard sample pairs while constraining the multiple preference distances to better separate the embedding space. Extensive experimental results demonstrate that the proposed method outperforms various state-of-the-art competitors on three public benchmark datasets.
SE-Prompt: Exploring Semantic Enhancement with Prompt Tuning for Relation Extraction

Compared to traditional supervised learning methods, utilizing prompt tuning for relation extraction tasks is a challenging endeavor in the real world. By inserting a template segment into the input, prompt tuning has proven effective for certain classification tasks. However, applying prompt tuning to relation extraction tasks, which involve mapping multiple words to a single label, poses challenges due to difficulties in precisely defining a template and mapping labels to the appropriate words. Prior approaches do not take full advantage of entities and have also overlooked the semantic connections between words in relation label. To address these limitations, we propose a semantic enhancement with prompt (SE-Prompt) which integrates entity and relation knowledge by incorporating two main contributions: semantic enhancement and subject-object relation refinement. These methods empower our model to effectively leverage relation labels and tap into the knowledge contained in pre-trained models. Our experiments on three datasets, under both fully supervised and low-resource settings demonstrate the effectiveness of our approach for relation extraction.
Graph Contrastive Learning with Hybrid Noise Augmentation for Recommendation

Recommendation System is one of the effective tools to solve the problem of information overload in the era of big data, but the data sparsity has greatly affected its performance. Recently, contrastive learning, has attracted great attention and is expected to solve this problem. However, most of the existing graph-based contrastive learning methods perturb the original graph for data enhancement, which may affect the recommended performance. Meanwhile, studies have shown that improving the uniformity of data distribution is more important than data augmentation by graph perturbation. In this paper, in order to improve the uniformity of the data distribution, we propose a Graph Contrastive Learning with Hybrid Noise Augmentation for Recommendation, which is abbreviated as GCLHANRec. Specifically, we add uniform distribution random noise to users and normal distribution random noise to items, to improve the data uniformity while increasing the user’s interest diversity for different items, thereby improving the accuracy and personalization degree of the recommendation system. Additionally, we propose Balanced Bayesian Personalized Ranking (BBPR) as the loss function for recommendation tasks, which is a modification of BPR to better make the model pay more attention to the difference between positive and negative samples, thus performing better in ranking tasks. We conducted extensive experiments using three datasets collected from actual environment, including Movielens, LastFM and Douban-book. The results show that our method outperforms several existing methods.
Breast Cancer Histopathology Image Classification Using Frequency Attention Convolution Network

The existing deep learning works mainly capture breast cancer histopathology image features in the spatial domain, and they rarely consider the frequency domain feature representation of histopathology images. According to the classical digital signal processing theory, frequency domain features may outperform spatial domain features in analyzing texture images. Motivated by this, we attempt to mine frequency domain features for the breast cancer histopathology image classification application, and further propose a novel frequency-attention convolutional network called SeFFT-Net by combining the Fourier transform with the channel attention mechanism. The core of SeFFT-Net consists of a newly constructed frequency-based squeeze and excitation (SeFFT) module, which first performs Fourier transform with residual construction to capture deep features in the frequency domain of histopathology images, followed by a squeeze-and-excitation attention operator to further enhance important frequency features. We extensively evaluate the proposed SeFFT-Net model on the public BreakHis breast cancer histopathology dataset, and it achieves the optimal image-level and patient-level classification accuracy of 98.67% and 98.16%, respectively. Meanwhile, ablation studies also well demonstrate the effectiveness of introducing frequency transforms for this medical image application.