OpenRaiser/Intern-Atlas huggingface dataset:
Dual-level hypergraph contrastive learning with adaptive temperature enhancement
Authors: Yiyue Qian, Tianyi Ma, Chuxu Zhang, Yanfang Ye
Venue: Conference WWW 2024
Year: 2024

Inspired by the success of graph contrastive learning, researchers have begun exploring the benefits of contrastive learning over hypergraphs. However, these works have the following limitations in modeling the high-order relationships over unlabeled data: (i) They primarily focus on maximizing the agreements among individual node embeddings while neglecting the capture of group-wise collective behaviors within hypergraphs; (ii) Most of them disregard the importance of the temperature index in discriminating contrastive pairs during contrast optimization. To address these limitations, we propose a novel dual-level Hy perG raph C ontrastive L earning framework with Ad aptive T emperature (HyGCL-AdT ) to boost contrastive learning over hypergraphs. Specifically, unlike most works that merely maximize the agreement of node embeddings in hypergraphs, we propose a dual-level contrast mechanism that not only captures the individual node behaviors in a local context but also models the group-wise collective behaviors of nodes within hyperedges from a community perspective. Besides, we design an adaptive temperature-enhanced contrastive optimization to improve the discrimination ability between contrastive pairs. Empirical experiments conducted on seven benchmark hypergraphs demonstrate that HyGCL-AdT exhibits excellent effectiveness compared to state-of-the-art baseline models. The source code is available at \hrefhttps://github.com/graphprojects/HyGCL-AdT https://github.com/graphprojects/HyGCL-AdT.
Undo: Effective and accurate unlearning method for deep neural networks
Authors: Sangyong Lee, Simon S. Woo
Venue: Conference CIKM 2023
Year: 2023

Machine learning has evolved through extensive data usage, including personal and private information. Regulations like GDPR highlight the "Right to be forgotten" for user and data privacy. Research in machine unlearning aims to remove specific data from pre-trained models. We introduce a novel two-step unlearning method, UNDO. First, we selectively disrupt the decision boundary of forgetting data at the coarse-grained level. However, this can also inadvertently affect the decision boundary of other remaining data, lowering the overall performance of classification task. Hence, we subsequently repair and refining the decision boundary for each class at the fine-grained level by introducing a loss for maintain the overall performance, while completely removing the class. Our approach is validated through experiments on two datasets, outperforming other methods in effectiveness and efficiency.
Incentive and dynamic client selection for federated unlearning
Authors: Yijing Lin, Zhipeng Gao, Hongyang Du, Dusit Niyato, Jiawen Kang, Xiaoyuan Liu
Venue: Conference WWW 2024
Year: 2024

With the development of AI-Generated Content (AIGC), data is becoming increasingly important, while the right of data to be forgotten, which is defined in the General Data Protection Regulation (GDPR) and permits data owners to remove information from AIGC models, is also arising. To protect this right in a distributed manner corresponding to federated learning, federated unlearning is employed to eliminate history model updates and unlearn the global model to mitigate data effects from the targeted clients intending to withdraw from training tasks. To diminish centralization failures, the hierarchical federated framework that is distributed and collaborative can be integrated into the unlearning process, wherein each cluster can support multiple AIGC tasks. However, two issues remain unexplored in current federated unlearning solutions: 1) getting remaining clients, those not withdraw from the task, to join the unlearning process, which demands additional resources and notably has fewer benefits than federated learning, particularly in achieving the original performance via alternative unlearning processes and 2) exploring mechanisms for dynamic unlearning in the selection of remaining clients possessing unbalanced data to avoid starting the unlearning from scratch. We initially consider a two-level incentive and unlearning mechanism to address the aforementioned challenges. At the lower level, we utilize evolutionary game theory to model the dynamic participation process, aiming to attract remaining clients to participate in retraining tasks. At the upper level, we integrate deep reinforcement learning into federated unlearning to dynamically select remaining clients to join the unlearning process to mitigate the bias introduced by the unbalanced data distribution among clients. Experimental results demonstrate that the proposed mechanisms outperform comparative methods, enhancing utilities and improving accuracy.
Topology-driven multi-view clustering via tensorial refined sigmoid rank minimization
Authors: Zhibin Gu, Zhendong Li, Songhe Feng
Venue: Conference KDD 2024
Year: 2024

Benefiting from the effective exploitation of the high-order correlations across multiple views, tensor-based multi-view clustering (TMVC) has garnered considerable attention in recent years. Nevertheless, prior TMVC techniques commonly involve assembling multiple view-specific spatial similarity graphs into a three-dimensional tensor, overlooking the intrinsic topological structure essential for precise clustering of data within a manifold. Additionally, mainstream techniques are constrained by equally shrinking all singular values to recover a low-rank tensor, limiting their capacity to distinguish significant variations among different singular values. In this investigation, we present an innovative TMVC framework termed toPology-driven multi-view clustering viA refined teNsorial sigmoiD rAnk minimization (PANDA ). Specifically, PANDA extracts view-specific topological structures from Euclidean graphs and intricately integrates them into a low-rank three-dimensional tensor, facilitating the concurrent utilization of intra-view topological connectivity and inter-view high-order correlations. Moreover, we develop a refined sigmoid function as the tighter surrogate to tensor rank, enabling the exploration of significant information of heterogeneous singular values. Meanwhile, the topological structures are merged into a unified structure with varying weights, associated with a connectivity constraint, empowering the significant divergence among views and the explicit cluster structure of the target graph are simultaneously leveraged. Extensive experiments demonstrate the superiority of PANDA, outperforming SOTA methods.
Graph contrastive learning reimagined: Exploring universality
Authors: Jiaming Zhuo, Can Cui, Kun Fu, Bingxin Niu, Dongxiao He, Chuan Wang, Yuanfang Guo, Zhen Wang, Xiaochun Cao, Liang Yang
Venue: Conference WWW 2024
Year: 2024

Real-world graphs exhibit diverse structures, including homophilic and heterophilic patterns, necessitating the development of a universal Graph Contrastive Learning (GCL) framework. Nonetheless, the existing GCLs, especially those with a local focus, lack universality due to the mismatch between the input graph structure and the homophily assumption for two primary components of GCLs. Firstly, the encoder, commonly Graph Convolution Network (GCN), operates as a low-pass filter, which assumes the input graph to be homophilic. This makes it challenging to aggregate features from neighbor nodes of the same class on heterophilic graphs. Secondly, the local positive sampling regards neighbor nodes as positive samples, which is inspired by the homophily assumption. This results in feature similarity amplification for the samples from the different classes (i.e., FALSE positive samples). Therefore, it is crucial to feed the encoder and positive sampling of GCLs with homophilic graph structures. This paper presents a novel GCL framework, named gRaph cOntraStive Exploring uNiversality (ROSEN), designed to achieve this objective. Specifically, ROSEN equips a local graph structure inference module, utilizing the Block Diagonal Property (BDP) of the affinity matrix extracted from node ego networks. This module can generate the homophilic graph structure by selectively removing disassortative edges. Extensive evaluations validate the effectiveness and universality of ROSEN across node classification and node clustering tasks.
Modeling diverse chemical reactions for single-step retrosynthesis via discrete latent variables
Authors: Huarui He, Jie Wang, Yunfei Liu, Feng Wu
Venue: Conference CIKM 2022
Year: 2022

Single-step retrosynthesis is the cornerstone of retrosynthesis planning, which is a crucial task for computer-aided drug discovery. The goal of single-step retrosynthesis is to identify the possible reactants that lead to the synthesis of the target product in one reaction. By representing organic molecules as canonical strings, existing sequence-based retrosynthetic methods treat the product-to-reactant retrosynthesis as a sequence-to-sequence translation problem. However, most of them struggle to identify diverse chemical reactions for a desired product due to the deterministic inference, which contradicts the fact that many compounds can be synthesized through various reaction types with different sets of reactants. In this work, we aim to increase reaction diversity and generate various reactants using discrete latent variables. We propose a novel sequence-based approach, namely RetroDVCAE, which incorporates conditional variational autoencoders into single-step retrosynthesis and associates discrete latent variables with the generation process. Specifically, RetroDVCAE uses the Gumbel-Softmax distribution to approximate the categorical distribution over potential reactions and generates multiple sets of reactants with the variational decoder. Experiments demonstrate that RetroDVCAE outperforms state-of-the-art baselines on both benchmark dataset and homemade dataset. Both quantitative and qualitative results show that RetroDVCAE can model the multi-modal distribution over reaction types and produce diverse reactant candidates.
Incorporating external knowledge for evidence-based fact verification
Authors: Anab Maulana Barik, Wynne Hsu, Mong-Li Lee
Venue: Conference WWW 2022
Year: 2022

Existing fact verification methods employ pre-trained language models such as BERT for the contextual representation of evidence sentences. However, such representations do not take into account commonsense knowledge and these methods often conclude that there is not enough information to predict whether a claim is supported or refuted by the evidence sentences. In this work, we propose a framework called CGAT that incorporates external knowledge from ConceptNet to enrich the contextual representations of evidence sentences. We employ graph attention models to propagate the information among the evidence sentences before predicting the veracity of the claim. Experiment results on the benchmark FEVER dataset and UKP Snopes Corpus indicate that the proposed approach leads to higher accuracy and FEVER score compared to state-of-the-art claim verification methods.
BrainNet: Epileptic wave detection from SEEG with hierarchical graph diffusion learning
Authors: Junru Chen, Yang Yang 0009, Tao Yu 0006, Yingying Fan, Xiaolong Mo, Carl Yang 0001
Venue: Conference KDD 2022
Year: 2022

Epilepsy is one of the most serious neurological diseases, affecting 1-2% of the world's population. The diagnosis of epilepsy depends heavily on the recognition of epileptic waves, i.e., disordered electrical brainwave activity in the patient's brain. Existing works have begun to employ machine learning models to detect epileptic waves via cortical electroencephalogram (EEG), which refers to brain data obtained from a noninvasive examination performed on the patient's scalp surface to record electrical activity in the brain. However, the recently developed stereoelectrocorticography (SEEG) method provides information in stereo that is more precise than conventional EEG, and has been broadly applied in clinical practice. Therefore, in this paper, we propose the first data-driven study to detect epileptic waves in a real-world SEEG dataset. While offering new opportunities, SEEG also poses several challenges. In clinical practice, epileptic wave activities are considered to propagate between different regions in the brain. These propagation paths, also known as the epileptogenic network, are deemed to be a key factor in the context of epilepsy surgery. However, the question of how to extract an exact epileptogenic network for each patient remains an open problem in the field of neuroscience. Moreover, the nature of epileptic waves and SEEG data inevitably leads to extremely imbalanced labels and severe noise. To address these challenges, we propose a novel model (BrainNet) that jointly learns the dynamic diffusion graphs and models the brain wave diffusion patterns. In addition, our model effectively aids in resisting label imbalance and severe noise by employing several self-supervised learning tasks and a hierarchical framework. By experimenting with the extensive real SEEG dataset obtained from multiple patients, we find that BrainNet outperforms several latest state-of-the-art baselines derived from time-series analysis.
Active model adaptation under unknown shift
Authors: Jie-Jing Shao, Yunlu Xu, Zhanzhan Cheng, Yufeng Li 0008
Venue: Conference KDD 2022
Year: 2022

Successful machine learning typically relies on fixed data distribution. However, due to unforeseen situations in the open world, distribution shift often occurs in applications. For instance, in the image recognition task, an unpredictable distributional shift may occur due to changes in background or lighting. Furthermore, to alleviate the harm of distribution shift, the resource budget is not infinite and often constrained. To cope with such a novel problem Resource Constrained Adaptation under Unknown Shift, in this paper we study active model adaptation both theoretically and empirically. First, we present a generalization analysis of active model adaptation for distribution shift. In theory, we show that active model adaptation could improve the generalization error from O(1/N) to O(1/N), with only a few queried samples. Second, based on the theoretical analysis, we present a systemic solution Auto, consisting of three sub-steps, that is, distribution tracking, sample selection and model adaptation. Specifically, we design a shifted distribution detection module to locate the distributional shifted samples. To fit the labeling budget, we employ a core-set algorithm to enhance the informativeness of the selected samples. Finally, we update the model through the newly queried labeled data. We conduct empirical studies of nine existing active strategies on diverse real world data sets and the results show that Auto could remarkably outperform all the baselines.
Enhancing sequential recommendation via LLM-based semantic embedding learning
Authors: Jun Hu, Wenwen Xia, Xiaolu Zhang, Chilin Fu, Weichang Wu, Zhaoxin Huan, Ang Li, Zuoli Tang, Jun Zhou
Venue: Conference WWW 2024
Year: 2024

Sequential recommendation systems (SRS) are crucial in various applications as they enable users to discover relevant items based on their past interactions. Recent advancements involving large language models (LLMs) have shown significant promise in addressing intricate recommendation challenges. However, these efforts exhibit certain limitations. Specifically, directly extracting representations from an LLM based on items' textual features and feeding them into a sequential model hold no guarantee that the semantic information of texts could be preserved in these representations. Additionally, concatenating textual descriptions of all items in an item sequence into a long text and feeding it into an LLM for recommendation results in lengthy token sequences, which largely diminishes the practical efficiency. In this paper, we introduce SAID, a framework that utilizes LLMs to explicitly learn Semantically Aligned item ID embeddings based on texts. For each item, SAID employs a projector module to transform an item ID into an embedding vector, which will be fed into an LLM to elicit the exact descriptive text tokens accompanied by the item. The item embeddings are forced to preserve fine-grained semantic information of textual descriptions. Further, the learned embeddings can be integrated with lightweight downstream sequential models for practical recommendations. In this way, SAID circumvents lengthy token sequences in previous works, reducing resources required in industrial scenarios and also achieving superior recommendation performance. Experiments on six public datasets demonstrate that SAID outperforms baselines by about 5% to 15% in terms of NDCG@10. Moreover, SAID has been deployed in Alipay's online advertising platform, achieving a 3.07% relative improvement of cost per mille (CPM) over baselines, with an online response time of under 20 milliseconds.
Building an Information Retrieval test Collection for Spontaneous Conversational Speech
Authors: Douglas W. Oard, Dagobert Soergel, David S. Doermann, Xiaoli Huang, G. Craig Murray, Jianqiang Wang, Bhuvana Ramabhadran, Martin Franz, Samuel Gustman, James Mayfield, Liliya Kharevych, Stephanie M. Strassel
Venue: Conference SIGIR 2004
Year: 2004

Test collections model use cases in ways that facilitate evaluation of information retrieval systems. This paper describes the use of search-guided relevance assessment to create a test collection for retrieval of spontaneous conversational speech. Approximately 10,000 thematically coherent segments were manually identified in 625 hours of oral history interviews with 246 individuals. Automatic speech recognition results, manually prepared summaries, controlled vocabulary indexing, and name authority control are available for every segment. Those features were leveraged by a team of four relevance assessors to identify topically relevant segments for 28 topics developed from actual user requests. Search-guided assessment yielded sufficient inter-annotator agreement to support formative evaluation during system development. Baseline results for ranked retrieval are presented to illustrate use of the collection.
SCAN: Designing and evaluating user interfaces to support retrieval from speech archives
Authors: Steve Whittaker, Julia Hirschberg, John Choi, Donald Hindle, Fernando C. N. Pereira, Amit Singhal
Venue: Conference SIGIR 1999
Year: 1999

Previous examinations of search in textual archives have assumed that users first retrieve a ranked set of documents relevant to their query, and then visually scan through these documents, to identify the information they seek. While document scanning is possible in text, it is much more laborious in speech archives, due to the inherently serial nature of speech. Yet, in developing tools for speech access, little attention has so far been paid to users' problems in scanning and extracting information from within "speech documents".
Computers and iPhones and mobile phones, oh my!: A logs-based comparison of search users on different devices
Authors: Maryam Kamvar, Melanie Kellar, Rajan Patel, Ya Xu
Venue: Conference WWW 2009
Year: 2009

We present a logs-based comparison of search patterns across three platforms: computers, iPhones and conventional phones. Our goal is to understand how search users differ from computer-based search users, and we focus heavily on the distribution and variability of tasks that users perform from each platform. The results suggest that search usage is much more focused for the average user than for the average computer-based user. However, search behavior on high-end phones resembles computer-based search behavior more so than search behavior. A wide variety of implications follow from these findings. First, there is no single search interface which is suitable for all phones. We suggest that for the higher-end phones, a close integration with the standard computer-based interface (in terms of personalization and available feature set) would be beneficial for the user, since these phones seem to be treated as an extension of the users' computer. For all other phones, there is a huge opportunity for personalizing the search experience for the user's mobile needs, as these users are likely to repeatedly search for a single type of information need on their phone.
An information retrieval approach to spelling suggestion
Authors: Sai Krishna, Prasad Pingali, Vasudeva Varma
Venue: Conference WWW 2010
Year: 2010

In this paper, we present a two-step language-independent spelling suggestion system. In the first step, candidate suggestions are generated using an Information Retrieval(IR) approach. In step two, candidate suggestions are re-ranked using a new string similarity measure that uses the length of the longest common substrings occurring at the beginning and end of the words. We obtained very impressive results by reranking candidate suggestions using the new similarity measure. The accuracy of first suggestion is 92.3%, 90.0% and 83.5% for Dutch, Danish and Bulgarian language datasets respectively.
Vocabulary Independent Spoken Term Detection
Authors: Jonathan Mamou, Bhuvana Ramabhadran, Olivier Siohan
Venue: Conference SIGIR 2007
Year: 2007

We are interested in retrieving information from speech data like broadcast news, telephone conversations and roundtable meetings. Today, most systems use large vocabulary continuous speech recognition tools to produce word transcripts; the transcripts are indexed and query terms are retrieved from the index. However, query terms that are not part of the recognizer's vocabulary cannot be retrieved, and the recall of the search is affected. In addition to the output word transcript, advanced systems provide also phonetic transcripts, against which query terms can be matched phonetically. Such phonetic transcripts suffer from lower accuracy and cannot be an alternative to word transcripts.We present a vocabulary independent system that can handle arbitrary queries, exploiting the information provided by having both word transcripts and phonetic transcripts. A speech recognizer generates word confusion networks and phonetic lattices. The transcripts are indexed for query processing and ranking purpose.The value of the proposed method is demonstrated by the relative high performance ofour system, which received the highest overall ranking for US English speech data in the recent NIST Spoken Term Detection evaluation.
An all-transputer visual Autobahn-autopilot/copilot
Authors: Ernst D. Dickmanns, Reinhold Behringer, Claus Brüdigam, Dirk Dickmanns, Frank Thomanek, Volker van Holt
Venue: Conference ICCV 1993
Year: 1993

An integrated system for visual road vehicle guidance has been created on transputer-hardware and partly validated in normal Autobahn traffic environments. The computer vision and dynamic scene understanding components are discussed. The system is capable of recognizing horizontal and vertical curvature of the lane, lane width, number of lanes, the state of the vehicle relative to the lane, and the relative state of two other objects by recursive monocular estimation following the 4-D approach as a unifying scheme for 3-D dynamic scene understanding through motion stereo. The architecture realized on transputer hardware is described. Experimental results with a 5-ton van are discussed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>