OpenRaiser/Intern-Atlas huggingface dataset:
BI-GCN: Bilateral Interactive Graph Convolutional Network for Recommendation
Authors: Yinan Zhang, Pei Wang, Congcong Liu, Xiwei Zhao, Hao Qi, Jie He, Junsheng Jin, Changping Peng, Zhangang Lin, Jingping Shao
Venue: CIKM
Year: 2023

Recently, Graph Convolutional Network (GCN) based methods have become novel state-of-the-arts for Collaborative Filtering (CF) based Recommender Systems. To obtain users' preferences over different items, it is a common practice to learn representations of users and items by performing embedding propagation on a user-item bipartite graph, and then calculate the preference scores based on the representations. However, in most existing algorithms, user/item representations are generated independently of target items/users. To address this problem, we propose a novel graph attention model named Bilateral Interactive GCN (BI-GCN), which introduces bilateral interactive guidance into each user-item pair and thus leads to target-aware representations for preference prediction. Specifically, to learn the user/item representation from its neighborhood, we assign higher attention weights to those neighbors similar to the target item/user. By this manner, we can obtain target-aware representations, i.e., the information of the target item/user is explicitly encoded in the corresponding user/item representation, for more precise matching. Extensive experiments on three benchmark datasets demonstrate the effectiveness and robustness of BI-GCN.
Unlocking the Potential of Deep Learning in Peak-Hour Series Forecasting
Authors: Zhenwei Zhang, Xin Wang, Jingyuan Xie, Heling Zhang, Yuantao Gu
Venue: CIKM
Year: 2023

Unlocking the potential of deep learning in Peak-Hour Series Forecasting (PHSF) remains a critical yet underexplored task in various domains. While state-of-the-art deep learning models excel in regular Time Series Forecasting (TSF), they struggle to achieve comparable results in PHSF. This can be attributed to the challenges posed by the high degree of non-stationarity in peak-hour series, which makes direct forecasting more difficult than standard TSF. Additionally, manually extracting the maximum value from regular forecasting results leads to suboptimal performance due to models minimizing the mean deficit. To address these issues, this paper presents Seq2Peak, a novel framework designed specifically for PHSF tasks, bridging the performance gap observed in TSF models. Seq2Peak offers two key components: the CyclicNorm pipeline to mitigate the non-stationarity issue and a simple yet effective trainable-parameter-free peak-hour decoder with a hybrid loss function that utilizes both the original series and peak-hour series as supervised signals. Extensive experimentation on publicly available time series datasets demonstrates the effectiveness of the proposed framework, yielding a remarkable average relative improvement of 37.7% across four real-world datasets for both transformer- and non-transformer-based TSF models.
Non-IID always Bad? Semi-Supervised Heterogeneous Federated Learning with Local Knowledge Enhancement
Authors: Chao Zhang, Fangzhao Wu, Jingwei Yi, Derong Xu, Yang Yu, Jindong Wang, Yidong Wang, Tong Xu, Xing Xie, Enhong Chen
Venue: CIKM
Year: 2023

Federated learning (FL) is important for privacy-preserving services by training models without collecting raw user data. Most FL algorithms assume all data is annotated, which is impractical due to the high cost of labeling data in real applications. To alleviate the reliance on labeled data, semi-supervised federated learning (SSFL) has been proposed to utilize unlabeled data on clients to improve model performance. However, most existing methods either have privacy issues which share models trained on other clients, or generate pseudo-labels for unlabeled local datasets with the global model, which is usually biased towards the global data distribution. The latter may lead to sub-optimal accuracy of pseudo-labels, due to the gap between the local data distribution and the global model, especially in non-IID settings. In this paper, we propose a semi-supervised heterogeneous federated learning method with local knowledge enhancement, called FedLoKe, which aims to train an accurate global model from both labeled and unlabeled local data with non-IID distributions. Specifically, in FedLoKe, the server maintains a global model to capture global data distribution, and each client learns a local model to capture local data distribution. Since the distribution captured by the local model is aligned with the local data distribution, we utilize it to generate high-accuracy pseudo-labels of the unlabeled dataset for global model training. To prevent the local model from severely overfitting the small number of local labeled data, we further use the exponential moving average and apply the global model to generate pseudo-labels for local modeling training. Experiments on four datasets show the effectiveness of FedLoKe. Our code is available at: https://github.com/zcfinal/FedLoKe.
Knowledge-enhanced Artificial Intelligence in Drug Discovery (KAIDD)
Authors: Qingpeng Zhang, Jiannan Yang
Venue: CIKM
Year: 2023

Artificial Intelligence (AI) in drug discovery is a rapidly evolving field that combines computational methods with biological knowledge and applications. Traditionally, the process of developing a new drug has been time-consuming and expensive, spanning several years and costing billions of dollars. The emergence of AI technologies offers the potential to significantly reduce both the timeline and cost involved in this critical endeavour. However, it is crucial to acknowledge that AI applications in pharmacy and drug discovery require a high degree of interpretability and transparency. The integration of domain knowledge into AI models becomes paramount to ensure the reliability and trustworthiness of the generated results. In light of these considerations, we propose a workshop on "Knowledge-enhanced Artificial Intelligence in Drug Discovery (KAIDD)." This workshop aims to explore the profound impact of incorporating various knowledge databases into the development of explainable AI models for drug discovery. Participants will have the opportunity to delve into cutting-edge research, methodologies, and practical applications that leverage the fusion of AI techniques with domain-specific knowledge. Authors of accepted papers will have the opportunity to submit extended versions of their work for a full-paper review process and potential publication in Philosophical Transactions of the Royal Society B.
MemoNet: Memorizing All Cross Features' Representations Efficiently via Multi-Hash Codebook Network for CTR Prediction
Authors: Pengtao Zhang, Junlin Zhang
Venue: CIKM
Year: 2023

New findings in natural language processing (NLP) demonstrate that the strong memorization capability contributes a lot to the success of Large Language Models (LLM). This inspires us to explicitly bring an independent memory mechanism into CTR ranking model to learn and memorize cross features' representations. In this paper, we propose multi-Hash Codebook NETwork (HCNet) as the memory mechanism for efficiently learning and memorizing representations of cross features in CTR tasks. HCNet uses a multi-hash codebook as the main memory place and the whole memory procedure consists of three phases: multi-hash addressing, memory restoring, and feature shrinking. We also propose a new CTR model named MemoNet which combines HCNet with a DNN backbone. Extensive experimental results on three public datasets and online test show that MemoNet reaches superior performance over state-of-the-art approaches. Besides, MemoNet shows scaling law of large language model in NLP, which means we can enlarge the size of the codebook in HCNet to sustainably obtain performance gains. Our work demonstrates the importance and feasibility of learning and memorizing representations of cross features, which sheds light on a new promising research direction. The source code is in https://github.com/ptzhangAlg/RecAlg.
No Length Left Behind: Enhancing Knowledge Tracing for Modeling Sequences of Excessive or Insufficient Lengths
Authors: Moyu Zhang, Xinning Zhu, Chunhong Zhang, Feng Pan, Wenchen Qian, Hui Zhao
Venue: CIKM
Year: 2023

Knowledge tracing (KT) aims to predict students' responses to practices based on their historical question-answering behaviors. However, most current KT methods focus on improving overall AUC, leaving ample room for optimization in modeling sequences of excessive or insufficient lengths. As sequences get longer, computational costs will increase exponentially. Therefore, KT methods usually truncate sequences to an acceptable length, which makes it difficult for models on online service systems to capture complete historical practice behaviors of students with too long sequences. Conversely, modeling students with short practice sequences using most KT methods may result in overfitting due to limited observation samples. To address the above limitations, we propose a model called Sequence-Flexible Knowledge Tracing (SFKT). Specifically, to flexibly handle long sequences, SFKT introduces a total-term encoder to effectively model complete historical practice behaviors of students at an affordable computational cost. Additionally, to improve the prediction accuracy of students with short practice sequences, we introduce a contrastive learning task and data augmentation schema to improve the generality of modeling short sequences by constructing more learning objectives. Extensive experimental results show that SFKT achieves significant improvements over multiple benchmarks, demonstrating the value of exploring the modeling of sequences of excessive or insufficient lengths. Our code is available at https://github.com/zmy-9/SFKT.
FiBiNet++: Reducing Model Size by Low Rank Feature Interaction Layer for CTR Prediction
Authors: Pengtao Zhang, Zheng Zheng, Junlin Zhang
Venue: CIKM
Year: 2023

Click-Through Rate (CTR) estimation has become one of the most fundamental tasks in many real-world applications and various deep models have been proposed. Some research has proved that FiBiNet is one of the best performance models and outperforms all other models on Avazu dataset. However, the large model size of FiBiNet hinders its wider application. In this paper, we propose a novel FiBiNet++ model to redesign FiBiNet's model structure, which greatly reduces model size while further improves its performance. One of the primary techniques involves our proposed "Low Rank Layer" focused on feature interaction, which serves as a crucial driver of achieving a superior compression ratio for models. Extensive experiments on three public datasets show that FiBiNet++ effectively reduces non-embedding model parameters of FiBiNet by 12x to 16x on three datasets. On the other hand, FiBiNet++ leads to significant performance improvements compared to state-of-the-art CTR methods, including FiBiNet. The source code is in https://github.com/recommendation-algorithm/FiBiNet.
Counterfactual Monotonic Knowledge Tracing for Assessing Students' Dynamic Mastery of Knowledge Concepts
Authors: Moyu Zhang, Xinning Zhu, Chunhong Zhang, Wenchen Qian, Feng Pan, Hui Zhao
Venue: CIKM
Year: 2023

As the core of the Knowledge Tracking (KT) task, assessing students' dynamic mastery of knowledge concepts is crucial for both offline teaching and online educational applications. Since students' mastery of knowledge concepts is often unlabeled, existing KT methods focus on predicting students' responses to practices. However, purely predicting student responses without imposing specific constraints on hidden concept mastery values does not guarantee the accuracy of these intermediate values as concept mastery values. To address this issue, we propose a principled approach called Counterfactual Monotonic Knowledge Tracing (CMKT), which builds on the implicit paradigm described above by using a counterfactual assumption to constrain the evolution of students' mastery of knowledge concepts. Specifically, CMKT first assesses students' knowledge concept mastery value based on their historical practice sequences. Then, CMKT sets the answer of the most recent practice as the opposite of the actual answer and, based on this counterfactual answer, assesses the student's corresponding counterfactual knowledge mastery value. During the model training process, CMKT constrains the update of the student's knowledge states by ensuring that the two types of knowledge mastery values of students satisfy a fundamental educational theory, the monotonicity theory, to provide specific semantics for the assessed mastery values by the model. Finally, extensive experiments on five datasets demonstrate the superiority of CMKT over baseline models.
Highly-Optimized Forgetting for Creating Signature-Based Views of Ontologies
Authors: Yizheng Zhao
Venue: CIKM
Year: 2023

Uniform interpolation (UI) is a non-standard reasoning service that seeks to project an ontology down to its sub-signature --- given an ontology taking a certain signature, and a subset Σ of "relevant names'' of that signature, compute a new ontology, called a uniform interpolant, that uses only the relevant names while preserving the semantics of the relevant names in the uniform interpolant. UI is of great potential importance since it may be used in a variety of applications where suitable views of ontologies need to be computed. However, this potential can only be fully realized if a highly optimized method for computing such views exists. Previous research has shown that computing uniform interpolants of ELH-ontologies is a computationally extremely hard problem --- a finite uniform interpolant does not always exist for ELH, and if it exists, then there exists one of at most triple exponential size in terms of the original ontology, and that, in the worst case, no shorter interpolant exists. Despite the inherent difficulty of the problem, in this paper, we present a highly optimized forgetting method for computing uniform interpolants of ELH-ontologies, and show however that, with good reduction and inference strategies, such uniform interpolants can be efficiently computed. The method is an improvement of the one presented in our previous work. What sets it apart is its flexibility to treat concept names of different types differently, effectively cutting down on the inferences involved. This treatment is primarily driven by the polarities of the concept names within an ontology. A comprehensive evaluation with a prototypical implementation of the method shows >95% average success rates over two popular benchmark datasets and demonstrates a clear computational advantage over state-of-the-art systems.
Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification
Authors: Chenyu Zhao, Yunjiang Jiang, Yiming Qiu, Han Zhang, Wen-Yun Yang
Venue: CIKM
Year: 2023

Retrieval augmentation, which enhances downstream models by a knowledge retriever and an external corpus instead of by merely increasing the number of model parameters, has been successfully applied to many natural language processing(NLP) tasks such as text classification, question answering and so on. However, existing methods that separately or asynchronously train the retriever and downstream model mainly due to the non-differentiability between the two parts, usually lead to degraded performance compared to end-to-end joint training. In this paper, we propose Differentiable Retrieval Augmentation via Generative lANguage modeling(Dragan), to address this problem by a novel differentiable reformulation. We demonstrate the effectiveness of our proposed method on a challenging NLP task in e-commerce search, namely query intent classification. Both the experimental results and ablation study show that the proposed method significantly and reasonably improves the state-of-the-art baselines on both offline evaluation and online A/B test.
Sequential Recommendation via an Adaptive Cross-domain Knowledge Decomposition
Authors: Chuang Zhao, Xinyu Li, Ming He, Hongke Zhao, Jianping Fan
Venue: CIKM
Year: 2023

Cross-domain recommendation, as an intelligent machine to alleviate data sparsity and cold start problems, has attracted extensive attention from scholars. Existing cross-domain recommendation frameworks usually leverage overlapping entities for knowledge transfer, the most popular of which are information aggregation and consistency maintenance. Despite decent improvements, the neglect of dynamic perspectives, the presence of confounding factors, and the disparities in domain properties inevitably constrain model performance. In view of this, this paper proposes a sequential recommendation framework via adaptive cross-domain knowledge decomposition, namely ARISEN, which focuses on employing adaptive causal learning to improve recommendation performance. Specifically, in order to facilitate sequence transfer, we align the user's behaviour sequences in the source domain and target domain according to the timestamps, expecting to use the abundant semantics of the former to augment the information of the latter. Regarding confounding factor removal, we introduce the causal learning technique and promote it as an adaptive representation decomposition framework on the basis of instrumental variables. For the sake of alleviating the impact of domain disparities, this paper endeavors to employ two mutually orthogonal transformation matrices for information fusion. Extensive experiments and detailed analyzes on large industrial and public data sets demonstrate that our framework can achieve substantial improvements over state-of-the-art algorithms.
HTMapper: Bidirectional Head-Tail Mapping for Nested Named Entity Recognition
Authors: Jin Zhao, Zhixu Li, Yanghua Xiao, Jiaqing Liang, Jingping Liu
Venue: CIKM
Year: 2023

Nested named entity recognition (Nested NER) aims to identify entities with nested structures from the given text, which is a fundamental task in Natural Language Processing. The region-based approach is the current mainstream approach, which first generates candidate spans and then classifies them into predefined categories. However, this method suffers from several drawbacks, including over-reliance on span representation, vulnerability to unbalanced category distribution, and inaccurate span boundary detection. To address these problems, we propose to model the nested NER problem into a head-tail mapping problem, namely, HTMapper, which detects head boundaries first and then models a conditional mapping from head to tail under a given category. Based on this mapping, we can find corresponding tails under different categories for each detected head by enumerating all entity categories. Our approach directly models the head boundary and tail boundary of entities, avoiding over-reliance on the span representation. Additionally, Our approach utilizes category information as an indicator signal to address the imbalance of category distribution during category prediction. Furthermore, our approach enhances the detection of span boundaries by capturing the correlation between head and tail boundaries. Extensive experiments on three nested NER datasets and two flat NER datasets demonstrate that our HTMapper achieves excellent performance with F1 scores of 89.09%, 88.30%, 81.57% on ACE2004,ACE2005, GENIA, and 94.26%, 91.40% on CoNLL03, OntoNotes, respectively.
GCformer: An Efficient Solution for Accurate and Scalable Long-Term Multivariate Time Series Forecasting
Authors: Yanjun Zhao, Ziqing Ma, Tian Zhou, Mengni Ye, Liang Sun, Yi Qian
Venue: CIKM
Year: 2023

Transformer-based models have emerged as promising tools for time series forecasting. However, these models cannot make accurate prediction for long input time series. On the one hand, they failed to capture long-range dependency within time series data. On the other hand, the long input sequence usually leads to large model size and high time complexity. To address these limitations, we present GCformer, which combines a structured global convolutional branch for processing long input sequences with a local Transformer-based branch for capturing short, recent signals. A cohesive framework for a global convolution kernel has been introduced, utilizing three distinct parameterization methods. The selected structured convolutional kernel in the global branch has been specifically crafted with sublinear complexity, thereby allowing for the efficient and effective processing of lengthy and noisy input signals. Empirical studies on six benchmark datasets demonstrate that GCformer outperforms state-of-the-art methods, reducing MSE error in multivariate time series benchmarks by 4.38% and model parameters by 61.92%. In particular, the global convolutional branch can serve as a plug-in block to enhance the performance of other models, with an average improvement of 31.93%, including various recently published Transformer-based models. Our code is publicly available at https://github.com/Yanjun-Zhao/GCformer.
Geometric Graph Learning for Protein Mutation Effect Prediction
Authors: Kangfei Zhao, Yu Rong, Biaobin Jiang, Jianheng Tang, Hengtong Zhang, Jeffrey Xu Yu, Peilin Zhao
Venue: CIKM
Year: 2023

Proteins govern a wide range of biological systems. Evaluating the changes in protein properties upon protein mutation is a fundamental application of protein design, where modeling the 3D protein structure is a principal task for AI-driven computational approaches. Existing deep learning (DL) approaches represent the protein structure as a 3D geometric graph and simplify the graph modeling to different degrees, thereby failing to capture the low-level atom patterns and high-level amino acid patterns simultaneously. In addition, limited training samples with ground truth labels and protein structures further restrict the effectiveness of DL approaches. In this paper, we propose a new graph learning framework, Hierarchical Graph Invariant Network (HGIN), a fine-grained and data-efficient graph neural encoder for encoding protein structures and predicting the mutation effect on protein properties. For fine-grained modeling, HGIN hierarchically models the low-level interactions of atoms and the high-level interactions of amino acid residues by Graph Neural Networks. For data efficiency, HGIN preserves the invariant encoding for atom permutation and coordinate transformation, which is an intrinsic inductive bias of property prediction that bypasses data augmentations. We integrate HGIN into a Siamese network to predict the quantitative effect on protein properties upon mutations. Our approach outperforms 9 state-of-the-art approaches on 3 protein datasets. More inspiringly, when predicting the neutralizing ability of human antibodies against COVID-19 mutant viruses, HGIN achieves an absolute improvement of 0.23 regarding the Spearman coefficient.