OpenRaiser/Intern-Atlas huggingface dataset:
Uncertainty-Aware Pre-Trained Foundation Models for Patient Risk Prediction via Gaussian Process
Authors: Jiaying Lu, Shifan Zhao, Wenjing Ma, Hui Shao, Xiao Hu, Yuanzhe Xi, Carl Yang
Venue: WWW
Year: 2024

Patient risk prediction models are crucial as they enable healthcare providers to proactively identify and address potential health risks. Large pre-trained foundation models offer remarkable performance in risk prediction tasks by analyzing multimodal patient data. However, a notable limitation of pre-trained foundation models lies in their deterministic predictions (i.e., lacking the ability to acknowledge uncertainty). We propose Gaussian Process-based foundation models to enable the generation of accurate predictions with instance-level uncertainty quantification, thus allowing healthcare professionals to make more informed and cautious decisions. Our proposed approach is principled and architecture-agnostic. Experimental results show that our proposed approach achieves competitive performance on classical classification metrics. Moreover, we observe that the accuracy of certain predictions is much higher than that of the uncertain ones, which validates the uncertainty awareness of our proposed method. Therefore, healthcare providers can trust low-uncertainty predictions and conduct more comprehensive investigations on high-uncertainty predictions, ultimately enhancing patient outcomes with less expert intervention.
Cluster Anchor Regularization to Alleviate Popularity Bias in Recommender Systems
Authors: Bo Chang, Changping Meng, He Ma, Shuo Chang, Yang Gu, Yajun Peng, Jingchen Feng, Yaping Zhang, Shuchao Bi, Ed H. Chi, Minmin Chen
Venue: WWW
Year: 2024

Recommender systems are essential for finding personalized content for users on online platforms. These systems are often trained on historical user interaction data, which collects user feedback on system recommendations. This creates a feedback loop leading to popularity bias; popular content is over-represented in the data, better learned, and thus recommended even more. Less popular content struggles to reach its potential audiences. Popularity bias limits the diversity of content that users are exposed to, and makes it harder for new creators to gain traction. Existing methods to alleviate popularity bias tend to trade off the performance of popular items. In this work, we propose a new method for alleviating popularity bias in recommender systems, called the cluster anchor regularization, which partitions the large item corpus into hierarchical clusters, and then leverages the cluster information of each item to facilitate transfer learning from head items to tail items. Our results demonstrate the effectiveness of the proposed method with offline analyses and live experiments on a large-scale industrial recommendation platform, where it significantly increases tail recommendation without hurting the overall user experience.
Towards Deeper Understanding of PPR-based Embedding Approaches: A Topological Perspective
Authors: Xingyi Zhang, Zixuan Weng, Sibo Wang
Venue: WWW
Year: 2024

Node embedding learns low-dimensional vectors for nodes in the graph. Recent state-of-the-art embedding approaches take Personalized PageRank (PPR) as the proximity measure and factorize the PPR matrix or its adaptation to generate embeddings. However, little previous work analyzes what information is encoded by these approaches, and how the information correlates with their superb performance in downstream tasks. In this work, we first show that state-of-the-art embedding approaches that factorize a PPR-related matrix can be unified into a closed-form framework. Then, we study whether the embeddings generated by this strategy can be inverted to better recover the graph topology information than random-walk based embeddings. To achieve this, we propose two methods for recovering graph topology via PPR-based embeddings, including the analytical method and the optimization method. Extensive experimental results demonstrate that the embeddings generated by factorizing a PPR-related matrix maintain more topological information, such as common edges and community structures, than that generated by random walks, paving a new way to systematically comprehend why PPR-based node embedding approaches outperform random walk-based alternatives in various downstream tasks. To the best of our knowledge, this is the first work that focuses on the interpretability of PPR-based node embedding approaches.
Contrastive Fingerprinting: A Novel Website Fingerprinting Attack over Few-shot Traces
Authors: Yi Xie, Jiahao Feng, Wenju Huang, Yixi Zhang, Xueliang Sun, Xiaochou Chen, Xiapu Luo
Venue: WWW
Year: 2024

Website Fingerprinting (WF) attacks enable passive adversaries to identify the website a user visits over encrypted or anonymized network connections. WF attacks based on deep learning have achieved high accuracy in identifying websites based on abundant training traffic traces per website. However, collecting large-scale and fresh traces is quite cost-consuming and unrealistic. Morevoer, these deep-learning-based WF attacks lack flexibility because they require a long bootstrap time for retraining when facing new traffic traces with different distributions or newly added monitored websites. This paper proposes a high-accuracy WF attack named Contrastive Fingerprinting (CF), which leverages contrastive learning and data augmentation over a few training traces. The results of extensive experiments on challenging datasets over few-shot traces demonstrate the high accuracy of the CF attack and its robustness against WF defenses. For example, when each monitored website only has 20 training traces, CF identifies monitored websites with a high accuracy of 90.4% in the closed-world scenario and distinguishes monitored websites with a high True Positive Rate of 91.2% in the open-world scenario. The experimental results also show that CF outperforms two existing WF attacks with few-shot traces under different network conditions in real-world applications.
Information Diffusion Meets Invitation Mechanism
Authors: Shiqi Zhang, Jiachen Sun, Wenqing Lin, Xiaokui Xiao, Yiqian Huang, Bo Tang
Venue: WWW
Year: 2024

The dissemination of information is a complex process that plays a crucial role in real-world applications, especially when intertwined with friend invitations and their ensuing responses. Traditional diffusion models, however, often do not adequately capture this invitation-aware diffusion (IAD), rendering inferior results. These models typically focus on describing the social influence process, i.e., how a user is informed by friends, but tend to overlook the subsequent behavioral changes that invitations might precipitate. To this end, we present the Independent Cascade with Invitation (ICI) model, which incorporates both the social influence process and multi-stage behavior conversions in IAD. We validate our design through an empirical study on in-game IAD. Furthermore, we conduct extensive experiments to evaluate the effectiveness of our proposal against 6 state-of-the-art models on 6 real-world datasets. In particular, we demonstrate that our solution can outperform the best competitor by up to 5× in cascade estimation and 17.2% in diffusion prediction. We deploy our proposal in the seed selection and friend ranking scenarios of Tencent's online games, where it achieves improvements of up to 170% and 20.3%, respectively.
Jointly Canonicalizing and Linking Open Knowledge Base via Unified Embedding Learning
Authors: Wei Shen, Binhan Yang, Yinan Liu
Venue: WWW
Year: 2024

Recent years have witnessed increasing attention on the semantic knowledge integration between curated knowledge bases (CKBs) and open knowledge bases (OKBs), which is non-trivial due to the intrinsically heterogeneous features involved in CKBs and OKBs. OKB canonicalization and OKB linking are regarded as two vital tasks to achieve the knowledge integration. Although these two tasks are inherently complementary with each other, previous studies just solve them separately or via superficial interaction. To address this issue, we propose CLUE, a novel framework that jointly encodes the OKB and CKB into a unified embedding space, to tackle OKB canonicalization and OKB linking simultaneously and make them benefit each other reciprocally. We design an expectation-maximization (EM) based approach to iteratively refine the unified embedding space via performing seed generation and embedding refinement alternately, by leveraging the deep interaction between OKB canonicalization and OKB linking. Curriculum learning is employed to yield high-quality canonicalization seeds and linking seeds adaptively, according to two elaborately designed metrics (i.e., a margin-based linking metric and an entropy-based cluster metric). A thorough experimental study over two public benchmark data sets demonstrates that our proposed CLUE consistently outperforms state-of-the-art baselines for the task of OKB canonicalization (resp. OKB linking) in terms of average F1 (resp. accuracy).
User Distribution Mapping Modelling with Collaborative Filtering for Cross Domain Recommendation
Authors: Weiming Liu, Chaochao Chen, Xinting Liao, Mengling Hu, Jiajie Su, Yanchao Tan, Fan Wang
Venue: WWW
Year: 2024

User cold-start recommendation aims to provide accurate items for the newly joint users and is a hot and challenging problem. Nowadays as people participant in different domains, how to recommend items in the new domain for users in an old domain has become more urgent. In this paper, we focus on the Dual Cold-Start Cross Domain Recommendation (Dual-CSCDR) problem. That is, providing the most relevant items for new users on the source and target domains. The prime task in Dual-CSCDR is to properly model user-item rating interactions and map user expressive embeddings across domains. However, previous approaches cannot solve Dual-CSCDR well, since they separate the collaborative filtering and distribution mapping process, leading to the error superimposition issue. Moreover, most of these methods fail to fully exploit the cross-domain relationship among large number of non-overlapped users, which strongly limits their performance. To fill this gap, we propose User Distribution Mapping model with Collaborative Filtering (UDMCF), a novel end-to-end cold-start cross-domain recommendation framework for the Dual-CSCDR problem. UDMCF includes two main modules, i.e., rating prediction module and distribution alignment module. The former module adopts one-hot ID vectors and multi-hot historical ratings for collaborative filtering via a contrastive loss. The latter module contains overlapped user embedding alignment and general user subgroup distribution alignment. Specifically, we innovatively propose unbalance distribution optimal transport with typical subgroup discovering algorithm to map the whole user distributions. Our empirical study on several datasets demonstrates that UDMCF significantly outperforms the state-of-the-art models under the Dual-CSCDR setting.
Heterogeneous Knowledge Grounding for Medical Question Answering with Retrieval Augmented Large Language Model
Authors: Wenting Zhao, Zhongfen Deng, Shweta Yadav, Philip S. Yu
Venue: WWW
Year: 2024

The Large Language Model (LLM) is renowned for its ability to encode a vast amount of general domain knowledge, enabling it to excel in question-answering, dialogue systems, and summarization tasks. However, the medical domain presents a unique challenge to LLM due to the distribution of medical knowledge, which follows a long-tail pattern. Existing approaches address this challenge by injecting medical knowledge into LLM through single sources such as medical textbooks or medical knowledge bases. However, medical knowledge is distributed across multiple heterogeneous information sources. A medical question-answering system can enhance answer coverage and confidence by considering these diverse knowledge sources together. To bridge this gap, we propose a novel approach called Heterogeneous Knowledge Retrieval-Augmented LLM for medical domain question answering. Our experiments, conducted on the MedQA-USMLE dataset, demonstrate promising performance improvements. These results underscore the importance of harnessing heterogeneous knowledge sources in the medical domain.
Graph Contrastive Learning Meets Graph Meta Learning: A Unified Method for Few-shot Node Tasks
Authors: Hao Liu, Jiarui Feng, Lecheng Kong, Dacheng Tao, Yixin Chen, Muhan Zhang
Venue: WWW
Year: 2024

Graph Neural Networks (GNNs) have become popular tools for Graph Representation Learning (GRL). One fundamental problem is few-shot node classification. Most existing methods follow the meta learning paradigm, showing the ability of fast generalization to few-shot tasks. However, recent works indicate that graph contrastive learning combined with fine-tuning can significantly outperform meta learning methods. Despite the empirical success, there is limited understanding of the reasons behind it. In our study, we first identify two crucial advantages of contrastive learning over meta learning, including (1) the comprehensive utilization of graph nodes and (2) the power of graph augmentations. To integrate the strength of both contrastive learning and meta learning on the few-shot node classification tasks, we introduce a new paradigm-Contrastive Few-Shot Node Classification (COLA). Specifically, COLA identifies semantically similar nodes only from augmented graphs, enabling the construction of meta-tasks without label information. Therefore, COLA can incorporate all nodes to construct meta-tasks, reducing the risk of overfitting. Through extensive experiments, we validate the necessity of each component in our design and demonstrate that COLA achieves new state-of-the-art on all tasks.
HiFI: Hierarchical Fairness-aware Integrated Ranking with Constrained Reinforcement Learning
Authors: Yifan Liu, Wei Xia, Weiwen Liu, Menghui Zhu, Weinan Zhang, Ruiming Tang, Yong Yu
Venue: WWW
Year: 2024

Integrated ranking is a critical component in industrial recommendation platforms. It combines candidate lists from different upstream channels or sources and ranks them into an integrated list, which will be exposed to users. During this process, to take responsibility for channel providers, the integrated ranking system needs to consider the exposure fairness among channels, which directly affects the opportunities of different channels being displayed to users. Besides, personalization also requires the integrated ranking system to consider the user's diverse preference on different channels besides items. Existing methods are hard to address both problems effectively. In this paper, we propose a Hierarchical Fairness-aware Integrated ranking (HiFI) framework. It contains a channel recommender and an item recommender, and the fairness constraint is on channels with constrained RL. We also design a gated attention layer (GAL) to effectively capture users' multi-faceted preferences. We compare HiFI with various baselines on public and industrial datasets, and HiFI achieves the state-of-the-art performance on both utility and fairness metrics. We also conduct an online A/B test to further validate the effectiveness of HiFI.
A Worldwide View on the Reachability of Encrypted DNS Services
Authors: Ruixuan Li, Baojun Liu, Chaoyi Lu, Haixin Duan, Jun Shao
Venue: WWW
Year: 2024

To protect user DNS privacy, four DNS over Encryption (DoE) protocols have been proposed, including DNS over TLS (DoT), DNS over HTTPS (DoH), DNS over QUIC (DoQ), and DNS over HTTP/3 (DoH3). Ensuring reachability stands as a prominent prerequisite for the proper functionality of these DoE protocols, driving considerable efforts in this domain. However, existing studies predominantly concentrate on a limited number of DoT/DoH domains or employ a restricted subset of vantage points (VPs). In this paper, we present the first comprehensive worldwide view of DoE service reachability. By collecting data from our 15-month-long scan, we elaborately built a list of 1302 operational DoE domains as measurement targets, 448 of which support IPv6. Then we performed 10M DoE over IPv4 (DoEv4) and 570K DoE over IPv6 (DoEv6) queries from 5K VPs over two months, encompassing 102 countries/regions. Our results reveal that the reachability of DoE services is poor in some countries/regions. Specifically, 592K (5.92%) DoEv4 queries and 28K (4.91%) DoEv6 queries are blocked. In countries/regions with strict Internet control, DoEv4 service blocking often occurs during TCP connection and QUIC version negotiation. Compared to DoEv4, the reachability of DoEv6 services is better. In particular, some DoE blocking policies target only specific IP addresses or DoE protocols, providing clients with the opportunity to access blocked DoE domains. Our study highlights the need for the DNS community to pay attention and improve the reachability of DoE services.
Improving Model Robustness against Adversarial Examples with Redundant Fully Connected Layer
Authors: Ziming Zhao, Zhaoxuan Li, Tingting Li, Jiongchi Yu, Fan Zhang, Rui Zhang
Venue: WWW
Year: 2024

Recent studies show that deep neural networks are extremely vulnerable, especially for adversarial examples of image classification models. However, the current defense technologies exhibit a series of limitations in terms of the adaptability of different attacks, the trade-off between clean-instance accuracy and robust one, as well as efficiency for train time overhead. To tackle these problems, we present a novel component, named redundant fully connected layer, which can be combined with existing model backbones in a pluggable manner. Specifically, we design a tailor-made loss function for it that leverages cosine similarity to maximize the difference and diversity of multiple fully connected parts. We conduct extensive experiments against 12 representative attacks (white-box and black-box), based on the popular dataset. The empirical evaluations show that our scheme realizes significant outcomes against various attacks with negligible additional training overhead, while hardly bringing collateral damage for clean-instance accuracy.
Trident: A Universal Framework for Fine-Grained and Class-Incremental Unknown Traffic Detection
Authors: Ziming Zhao, Zhaoxuan Li, Zhuoxue Song, Wenhao Li, Fan Zhang
Venue: WWW
Year: 2024

To detect unknown attack traffic, anomaly-based network intrusion detection systems (NIDSs) are widely used in Internet infrastructure. However, the security communities realize some limitations when they put most existing proposals into practice. The challenges are mainly concerned with (i) fine-grained emerging attack detection and (ii) incremental updates/adaptations. To tackle these problems, we propose to decouple the need for model capabilities by transforming known/new class identification issues into multiple independent one-class learning tasks. Based on the above core ideas, we develop Trident, a universal framework for fine-grained unknown encrypted traffic detection. It consists of three main modules, i.e., tSieve, tScissors, and tMagnifier are used for profiling traffic, determining outlier thresholds, and clustering respectively, each of which supports custom configuration. Using four popular datasets of network traces, we show that Trident significantly outperforms 16 state-of-the-art (SOTA) methods. Furthermore, a series of experiments (concept drift, overhead/parameter evaluation) demonstrate the stability, scalability, and practicality of Trident.
GEES: Enabling Location Privacy-Preserving Energy Saving in Multi-Access Edge Computing
Authors: Ziqi Wang, Xiaoyu Xia, Minhui Xue, Ibrahim Khalil, Minghui Liwang, Xun Yi
Venue: WWW
Year: 2024

The global deployment of the 5G network has led to a substantial increase in the deployment of edge servers to host web applications, catering to the growing demand for low service latency by edge web users. Yet, running edge servers 24/7 leads to enormous energy consumption and excessive carbon emissions. Energy-efficient edge resource provision is desired to achieve sustainable development goals in the new multi-access edge computing (MEC) architecture. Recently, several approaches have been proposed to solve the demand response problem for energy saving in cloud computing and MEC. However, accurate location information of edge web users should always be provided, which sacrifices users' privacy. To protect edge web users' location privacy while saving energy in MEC, we systematically formulate this location privacy-preserving edge demand response (LEDR) problem. To solve the LEDR problem effectively and efficiently, we propose a system named GEES by incorporating differential geo-obfuscation to secure user privacy while maximizing system utility and energy efficiency through inferences with theoretical analysis. Extensive and comprehensive experiments are conducted based on a synthetic real-world dataset, and the results demonstrate that GEES outperforms representative approaches by 23.02%, 31.47%, and 17.29% on average in terms of energy efficiency, user privacy and system utility.
Learning to Rewrite Prompts for Personalized Text Generation
Authors: Cheng Li, Mingyang Zhang, Qiaozhu Mei, Weize Kong, Michael Bendersky
Venue: WWW
Year: 2024

Facilitated by large language models (LLMs), personalized text generation has become a rapidly growing research direction. Most existing studies focus on designing specialized models for a particular domain, or they require fine-tuning the LLMs to generate personalized text. We consider a typical scenario in which the large language model, which generates personalized output, is frozen and can only be accessed through APIs. Under this constraint, all one can do is to improve the input text (i.e., text prompts) sent to the LLM, a procedure that is usually done manually. In this paper, we propose a novel method to automatically revise prompts for personalized text generation. The proposed method takes the initial prompts generated by a state-of-the-art, multistage framework for personalized generation and rewrites a few critical components that summarize and synthesize the personal context. The prompt rewriter employs a training paradigm that chains together supervised learning (SL) and reinforcement learning (RL), where SL reduces the search space of RL and RL facilitates end-to-end training of the rewriter. Using datasets from three representative domains, we demonstrate that the rewritten prompts outperform both the original prompts and the prompts optimized via supervised learning or reinforcement learning alone. In-depth analysis of the rewritten prompts shows that they are not only human readable, but also able to guide manual revision of prompts when there is limited resource to employ reinforcement learning to train the prompt rewriter, or when it is costly to deploy an automatic prompt rewriter for inference.