OpenRaiser/Intern-Atlas huggingface dataset:
Optimizing Smartphone App Usage Prediction: A Click-Through Rate Ranking Approach
Authors: Yuqi Zhang, Meiying Kang, Xiucheng Li, Yu Qiu, Zhijun Li 0002
Venue: KDD
Year: 2024

Over the past decade, smartphones have become indispensable personal mobile devices, experiencing a remarkable surge in software apps. These apps empower users to seamlessly connect with various internet services, such as social communication and online shopping. Accurately predicting smartphone app usage can effectively improve user experience and optimize resource utilization. However, existing models often treat app usage prediction as a classification problem, which suffers from issues of app usage imbalance and out-of-distribution (OOD) during deployment. To address these challenges, this paper proposes a novel click-through rate (CTR) ranking-based method for predicting app usage. By transforming the classification problem into a CTR problem, we can eliminate the negative impact of the app usage imbalance issue. To address the OOD issue during deployment, we generate the app click sequence and three types of discriminative features, which enable generalization on unseen apps. The app click sequence and the three types of features serve as inputs for training a CTR estimation model in the cloud, and the trained model is then deployed on the user's smartphone to predict the CTR for each installed app. The decision-making process involves ranking these CTR values and selecting the app with the highest CTR as the final prediction. Our method has been extensively tested with large-scale app usage data. The results demonstrate that our approach is able to outperform state-of-the-art methods, with improvements over 4.93% in top-3 accuracy and 6.64% in top-5 accuracy. It achieves approximately twice the accuracy in predicting apps with low usage frequencies in comparison to baseline methods. Our method has been successfully deployed on the app recommendation system of a leading smartphone manufacturer.
Joint Auction in the Online Advertising Market
Authors: Zhen Zhang 0053, Weian Li, Yahui Lei, Bingzhe Wang, Zhicheng Zhang, Qi Qi 0003, Qiang Liu, Xingxing Wang
Venue: KDD
Year: 2024

Online advertising is a primary source of income for e-commerce platforms. In the current advertising pattern, the oriented targets are the online store owners who are willing to pay extra fees to enhance the position of their stores. On the other hand, brand suppliers are also desirable to advertise their products in stores to boost brand sales. However, the currently used advertising mode cannot satisfy the demand of both stores and brand suppliers simultaneously. To address this, we innovatively propose a joint advertising model termed ''Joint Auction'', allowing brand suppliers and stores to collaboratively bid for advertising slots, catering to both their needs. However, conventional advertising auction mechanisms are not suitable for this novel scenario. In this paper, we propose JRegNet, a neural network architecture for the optimal joint auction design, to generate mechanisms that can achieve the optimal revenue and guarantee (near-)dominant strategy incentive compatibility and individual rationality. Finally, multiple experiments are conducted on synthetic and real data to demonstrate that our proposed joint auction significantly improves platform's revenue compared to the known baselines.
Knowledge Distillation with Perturbed Loss: From a Vanilla Teacher to a Proxy Teacher
Authors: Rongzhi Zhang, Jiaming Shen, Tianqi Liu 0002, Jialu Liu, Michael Bendersky, Marc Najork, Chao Zhang 0014
Venue: KDD
Year: 2024

Knowledge distillation is a popular technique to transfer knowledge from a large teacher model to a small student model. Typically, the student learns to imitate the teacher by minimizing the KL divergence of its output distribution with the teacher's output distribution. In this work, we argue that such a learning objective is sub-optimal because there exists a discrepancy between the teacher's output distribution and the ground truth label distribution. Therefore, forcing the student to blindly imitate the unreliable teacher output distribution leads to inferior performance. To this end, we propose a novel knowledge distillation objective PTLoss by first representing the vanilla KL-based distillation loss function via a Maclaurin series and then perturbing the leading-order terms in this series. This perturbed loss implicitly transforms the original teacher into a proxy teacher with a distribution closer to the ground truth distribution. We establish the theoretical connection between this "distribution closeness'' and the student model generalizability, which enables us to select the PTLoss's perturbation coefficients in a principled way. Extensive experiments on six public benchmark datasets demonstrate the effectiveness of PTLoss with teachers of different scales.
Item-Difficulty-Aware Learning Path Recommendation: From a Real Walking Perspective
Authors: Haotian Zhang 0007, Shuanghong Shen, Bihan Xu, Zhenya Huang, Jinze Wu, Jing Sha, Shijin Wang 0001
Venue: KDD
Year: 2024

Learning path recommendation aims to provide learners with a reasonable order of items to achieve their learning goals. Intuitively, the learning process on the learning path can be metaphorically likened to walking. Despite extensive efforts in this area, most previous methods mainly focus on the relationship among items but overlook the difficulty of items, which may raise two issues from a real walking perspective: (1) The path may be rough: When learners tread the path without considering item difficulty, it's akin to walking a dark, uneven road, making learning harder and dampening interest. (2) The path may be inefficient: Allowing learners only a few attempts on very challenging items before switching, or persisting with a difficult item despite numerous attempts without mastery, can result in inefficiencies in the learning journey. To conquer the above limitations, we propose a novel method named Difficulty-constrained Learning Path Recommendation (DLPR), which is aware of item difficulty. Specifically, we first explicitly categorize items into learning items and practice items, then construct a hierarchical graph to model and leverage item difficulty adequately. Then we design a Difficulty-driven Hierarchical Reinforcement Learning (DHRL) framework to facilitate learning paths with efficiency and smoothness. Finally, extensive experiments on three different simulators demonstrate our framework achieves state-of-the-art performance.
Natural Language Explainable Recommendation with Robustness Enhancement
Authors: Jingsen Zhang, Jiakai Tang, Xu Chen 0017, Wenhui Yu, Lantao Hu, Peng Jiang 0002, Han Li 0005
Venue: KDD
Year: 2024

Natural language explainable recommendation has become a promising direction to facilitate more efficient and informed user decisions. Previous models mostly focus on how to enhance the explanation accuracy. However, the robustness problem has been largely ignored, which requires the explanations generated for similar user-item pairs should not be too much different. Different from traditional classification problems, improving the robustness of natural languages has two unique characteristics: (1) Different token importances, that is, different tokens play various roles in representing the complete sentence, and the robustness requirements for predicting them should also be different. (2) Continuous token semantics, that is, the similarity of the output should be judged based on semantics, and the sequences without any token-level overlap may also be highly similar. Based on these characteristics, we formulate and solve a novel problem in the recommendation domain, that is, robust natural language explainable recommendation. To the best of our knowledge, it is the first time in this field. Specifically, we base our modeling on adversarial robust optimization and design four types of heuristic methods to modify the adversarial outputs with weighted token probabilities and synonym replacements. Furthermore, to consider the mutual influence between the above characteristics, we regard language generation as a decision-making problem and design a dual-policy reinforcement learning framework to improve the robustness of the generated languages. We conduct extensive experiments to demonstrate the effectiveness of our framework.
Heuristic Learning with Graph Neural Networks: A Unified Framework for Link Prediction
Authors: Juzheng Zhang, Lanning Wei, Zhen Xu 0007, Quanming Yao
Venue: KDD
Year: 2024

Link prediction is a fundamental task in graph learning, inherently shaped by the topology of the graph. While traditional heuristics are grounded in graph topology, they encounter challenges in generalizing across diverse graphs. Recent research efforts have aimed to leverage the potential of heuristics, yet a unified formulation accommodating both local and global heuristics remains undiscovered. Drawing insights from the fact that both local and global heuristics can be represented by adjacency matrix multiplications, we propose a unified matrix formulation to accommodate and generalize various heuristics. We further propose the Heuristic Learning Graph Neural Network (HL-GNN) to efficiently implement the formulation. HL-GNN adopts intra-layer propagation and inter-layer connections, allowing it to reach a depth of around 20 layers with lower time complexity than GCN. Extensive experiments on the Planetoid, Amazon, and OGB datasets underscore the effectiveness and efficiency of HL-GNN. It outperforms existing methods by a large margin in prediction performance. Additionally, HL-GNN is several orders of magnitude faster than heuristic-inspired methods while requiring only a few trainable parameters. The case study further demonstrates that the generalized heuristics and learned weights are highly interpretable.
Unified Dual-Intent Translation for Joint Modeling of Search and Recommendation
Authors: Yuting Zhang 0010, Yiqing Wu, Ruidong Han, Ying Sun 0006, Yongchun Zhu, Xiang Li 0067, Wei Lin 0022, Fuzhen Zhuang, Zhulin An, Yongjun Xu 0001
Venue: KDD
Year: 2024

Recommendation systems, which assist users in discovering their preferred items among numerous options, have served billions of users across various online platforms. Intuitively, users' interactions with items are highly driven by their unchanging inherent intents (e.g., always preferring high-quality items) and changing demand intents (e.g., wanting a T-shirt in summer but a down jacket in winter). However, both types of intents are implicitly expressed in recommendation scenario, posing challenges in leveraging them for accurate intent-aware recommendations. Fortunately, in search scenario, often found alongside recommendation on the same online platform, users express their demand intents explicitly through their query words. Intuitively, in both scenarios, a user shares the same inherent intent and the interactions may be influenced by the same demand intent. It is therefore feasible to utilize the interaction data from both scenarios to reinforce the dual intents for joint intent-aware modeling. But the joint modeling should deal with two problems: 1) accurately modeling users' implicit demand intents in recommendation; 2) modeling the relation between the dual intents and the interactive items. To address these problems, we propose a novel model named Unified Dual-Intents Translation for joint modeling of Search and Recommendation (UDITSR). To accurately simulate users' demand intents in recommendation, we utilize real queries from search data as supervision information to guide its generation. To explicitly model the relation among the triplet <inherent intent, demand intent, interactive item>, we propose a dual-intent translation propagation mechanism to learn the triplet in the same semantic space via embedding translations. Extensive experiments demonstrate that UDITSR outperforms SOTA baselines both in search and recommendation tasks.
Temporal Uplift Modeling for Online Marketing
Authors: Xin Zhang 0091, Kai Wang 0064, Zengmao Wang, Bo Du 0001, Shiwei Zhao, Runze Wu 0001, Xudong Shen, Tangjie Lv, Changjie Fan
Venue: KDD
Year: 2024

In recent years, uplift modeling, also known as individual treatment effect (ITE) estimation, has seen wide applications in online marketing, such as delivering one-time issuance of coupons or discounts to motivate users' purchases. However, complex yet more realistic scenarios involving multiple interventions over time on users are still rarely explored. The challenges include handling the bias from time-varying confounders, determining optimal treatment timing, and selecting among numerous treatments. In this paper, to tackle the aforementioned challenges, we present a temporal point process-based uplift model (TPPUM) that utilizes users' temporal event sequences to estimate treatment effects via counterfactual analysis and temporal point processes. In this model, marketing actions are considered as treatments, user purchases as outcome events, and how treatments alter the future conditional intensity function of generating outcome events as the uplift. Empirical evaluations demonstrate that our method outperforms existing baselines on both real-world and synthetic datasets. In the online experiment conducted in a discounted bundle recommendation scenario involving an average of 3 to 4 interventions per day and hundreds of treatment candidates, we demonstrate how our model outperforms current state-of-the-art methods in selecting the appropriate treatment and timing of treatment, resulting in a 3.6% increase in application-level revenue.
LLM4DyG: Can Large Language Models Solve Spatial-Temporal Problems on Dynamic Graphs?
Authors: Zeyang Zhang 0001, Xin Wang 0019, Ziwei Zhang 0001, Haoyang Li 0001, Yijian Qin, Wenwu Zhu 0001
Venue: KDD
Year: 2024

In an era marked by the increasing adoption of Large Language Models (LLMs) for various tasks, there is a growing focus on exploring LLMs' capabilities in handling web data, particularly graph data. Dynamic graphs, which capture temporal network evolution patterns, are ubiquitous in real-world web data. Evaluating LLMs' competence in understanding spatial-temporal information on dynamic graphs is essential for their adoption in web applications, which remains unexplored in the literature. In this paper, we bridge the gap via proposing to evaluate LLMs' spatial-temporal understanding abilities on dynamic graphs, to the best of our knowledge, for the first time. Specifically, we propose the LLM4DyG benchmark, which includes nine specially designed tasks considering the capability evaluation of LLMs from both temporal and spatial dimensions. Then, we conduct extensive experiments to analyze the impacts of different data generators, data statistics, prompting techniques, and LLMs on the model performance. Finally, we propose Disentangled Spatial-Temporal Thoughts (DST2) for LLMs on dynamic graphs to enhance LLMs' spatial-temporal understanding abilities. Our main observations are: 1) LLMs have preliminary spatial-temporal understanding abilities on dynamic graphs, 2) Dynamic graph tasks show increasing difficulties for LLMs as the graph size and density increase, while not sensitive to the time span and data generation mechanism, 3) the proposed DST2 prompting method can help to improve LLMs' spatial-temporal understanding abilities on dynamic graphs for most tasks. The data and codes are publicly available at Github.
RelKD 2024: The Second International Workshop on Resource-Efficient Learning for Knowledge Discovery
Authors: Chuxu Zhang, Dongkuan Xu, Kaize Ding, Jundong Li, Mojan Javaheripi, Subhabrata Mukherjee, Nitesh V. Chawla, Huan Liu 0001
Venue: KDD
Year: 2024

Modern machine learning techniques, particularly deep learning, have showcased remarkable efficacy across numerous knowledge discovery and data mining applications. However, the advancement of many of these methods is frequently impeded by resource constraint challenges in many scenarios, such as limited labeled data (data-level), small model size requirements in real-world computing platforms (model-level), and efficient mapping of the computations to heterogeneous target hardware (system-level). Addressing all these factors is crucial for effectively and efficiently deploying developed models across a broad spectrum of real-world systems, including large-scale social network analysis, recommendation systems, and real-time anomaly detection. Therefore, there is a critical need to develop efficient learning techniques to address the challenges posed by resource limitations, whether from data, model/algorithm, or system/hardware perspectives. The proposed international workshop on "<u>R</u>esource-<u>E</u>fficient <u>L</u>earning for <u>K</u>nowledge <u>D</u>iscovery (RelKD 2024)" will provide a great venue for academic researchers and industrial practitioners to share challenges, solutions, and future opportunities of resource-efficient learning.
Representation Learning of Geometric Trees
Authors: Zheng Zhang 0047, Allen Zhang 0005, Ruth Nelson, Giorgio A. Ascoli, Liang Zhao 0002
Venue: KDD
Year: 2024

Geometric trees are characterized by their tree-structured layout and spatially constrained nodes and edges, which significantly impacts their topological attributes. This inherent hierarchical structure plays a crucial role in domains such as neuron morphology and river geomorphology, but traditional graph representation methods often overlook these specific characteristics of tree structures. To address this, we introduce a new representation learning framework tailored for geometric trees. It first features a unique message passing neural network, which is both provably geometrical structure-recoverable and rotation-translation invariant. To address the data label scarcity issue, our approach also includes two innovative training targets that reflect the hierarchical ordering and geometric structure of these geometric trees. This enables fully self-supervised learning without explicit labels. We validate our method's effectiveness on eight real-world datasets, demonstrating its capability to represent geometric trees.
Geometric View of Soft Decorrelation in Self-Supervised Learning
Authors: Yifei Zhang, Hao Zhu 0010, Zixing Song, Yankai Chen 0001, Xinyu Fu 0004, Ziqiao Meng, Piotr Koniusz, Irwin King
Venue: KDD
Year: 2024

Contrastive learning, a form of Self-Supervised Learning (SSL), typically consists of an alignment term and a regularization term. The alignment term minimizes the distance between the embeddings of a positive pair, while the regularization term prevents trivial solutions and expresses prior beliefs about the embeddings. As a widely used regularization technique, soft decorrelation has been employed by several non-contrastive SSL methods to avoid trivial solutions. While the decorrelation term is designed to address the issue of dimensional collapse, we find that it fails to achieve this goal theoretically and experimentally. Based on such a finding, we extend the soft decorrelation regularization to minimize the distance between the covariance matrix and an identity matrix. We provide a new perspective on the geometric distance between positive definite matrices to investigate why the soft decorrelation cannot efficiently solve the dimensional collapse. Furthermore, we construct a family of loss functions utilizing the Bregman Matrix Divergence (BMD), with the soft decorrelation representing a specific instance within this family. We prove that a loss function (LogDet) in this family can solve the issue of dimensional collapse. Our novel loss functions based on BMD exhibit superior performance compared to the soft decorrelation and other baseline techniques, as demonstrated by experimental results on graph and image datasets.
Urban-Focused Multi-Task Offline Reinforcement Learning with Contrastive Data Sharing
Authors: Xinbo Zhao 0001, Yingxue Zhang 0002, Xin Zhang 0098, Yu Yang 0010, Yiqun Xie, Yanhua Li, Jun Luo 0007
Venue: KDD
Year: 2024

Enhancing diverse human decision-making processes in an urban environment is a critical issue across various applications, including ride-sharing vehicle dispatching, public transportation management, and autonomous driving. Offline reinforcement learning (RL) is a promising approach to learn and optimize human urban strategies (or policies) from pre-collected human-generated spatial-temporal urban data. However, standard offline RL faces two significant challenges: (1) data scarcity and data heterogeneity, and (2) distributional shift. In this paper, we introduce MODA - a Multi-Task Offline Reinforcement Learning with Contrastive Data Sharing approach. MODA addresses the challenges of data scarcity and heterogeneity in a multi-task urban setting through Contrastive Data Sharing among tasks. This technique involves extracting latent representations of human behaviors by contrasting positive and negative data pairs. It then shares data presenting similar representations with the target task, facilitating data augmentation for each task. Moreover, MODA develops a novel model-based multi-task offline RL algorithm. This algorithm constructs a robust Markov Decision Process (MDP) by integrating a dynamics model with a Generative Adversarial Network (GAN). Once the robust MDP is established, any online RL or planning algorithm can be applied. Extensive experiments conducted in a real-world multi-task urban setting validate the effectiveness of MODA. The results demonstrate that MODA exhibits significant improvements compared to state-of-the-art baselines, showcasing its capability in advancing urban decision-making processes. We also made our code available to the research community.
3rd Workshop on End-End Customer Journey Optimization
Authors: Shadow Zhao, Mert Bay, Anbang Xu, Neha Gupta
Venue: KDD
Year: 2024

Nowadays, while most machine learning research on customer journey optimization has focused on short-term success metrics such as click-through rates or optimal ad placement, there has been little consideration given to developing a coherent system for end-to-end customer journey optimization. Such a system would encompass all aspects of the customer experience, from presenting the right product value to the right users, to understanding a user's likelihood of conversion and long-term value to the platform, as well as their propensity for cross-selling and risk of churning. Currently, models and algorithms for customer journey optimization are often developed in isolation, leading to inefficiencies in modeling and data pipelines. Furthermore, the customer is often viewed as a collection of different entities by different organizational departments (such as marketing, sales, and finance), which can lead to additional friction in the customer experience. This workshop seeks to bridge the gap between academic researchers and industrial practitioners who are interested in building holistic solutions for end-to-end customer journey optimization. In addition, with the rising popularity of generative AI and LLM, we want to use this venue to exchange ideas regarding their applications in different stages of customer journey, and how the new technologies could help businesses achieve their KPIs.