OpenRaiser/Intern-Atlas huggingface dataset:
Graph neural networks beyond compromise between attribute and topology
Authors: Liang Yang, Wenmiao Zhou, Weihang Peng, Bingxin Niu, Junhua Gu, Chuan Wang, Xiaochun Cao, Dongxiao He
Venue: Conference WWW 2022
Year: 2022

Although existing Graph Neural Networks (GNNs) based on message passing achieve state-of-the-art, the over-smoothing issue, node similarity distortion issue and dissatisfactory link prediction performance can’t be ignored. This paper summarizes these issues as the interference between topology and attribute for the first time. By leveraging the recently proposed optimization perspective of GNNs, this interference is analyzed and ascribed to that the learned representation in GNNs essentially compromises between the topology and node attribute. To alleviate the interference, this paper attempts to break this compromise by proposing a novel objective function, which fits node attribute and topology with different representations and introduces mutual exclusion constraints to reduce the redundancy in both representations. The mutual exclusion employs the statistical dependence, which regards the representations from topology and attribute as the observations of two random variables, and is implemented with Hilbert-Schmidt Independence Criterion. Derived from the novel objective function, a novel GNN, i.e., Graph Neural Network Beyond Compromise (GNN-BC), is proposed to iteratively updates the representations of topology and attribute by simultaneously capturing semantic information and removing the common information, and the final representation is the concatenation of them. The performance improvements on node classification and link prediction demonstrate the superiority of GNN-BC on relieving the interference.
On directly mapping relational databases to RDF and OWL
Authors: Juan F. Sequeda, Marcelo Arenas, Daniel P. Miranker
Venue: Conference WWW 2012
Year: 2012

Mapping relational databases to RDF is a fundamental problem for the development of the Semantic Web. We present a solution, inspired by draft methods defined by the W3C where relational databases are directly mapped to RDF and OWL. Given a relational database schema and its integrity constraints, this direct mapping produces an OWL ontology, which, provides the basis for generating RDF instances. The semantics of this mapping is defined using Datalog. Two fundamental properties are information preservation and query preservation. We prove that our mapping satisfies both conditions, even for relational databases that contain null values. We also consider two desirable properties: monotonicity and semantics preservation. We prove that our mapping is monotone and also prove that no monotone mapping, including ours, is semantic preserving. We realize that monotonicity is an obstacle for semantic preservation and thus present a non-monotone direct mapping that is semantics preserving.
MR2: A benchmark for multimodal retrieval-augmented rumor detection in social media
Authors: Xuming Hu, Zhijiang Guo, Junzhe Chen, Lijie Wen, Philip S. Yu
Venue: Conference SIGIR 2023
Year: 2023

As social media platforms are evolving from text-based forums into multi-modal environments, the nature of misinformation in social media is also transforming accordingly. Misinformation spreaders have recently targeted contextual connections between the modalities e.g., text and image. However, existing datasets for rumor detection mainly focus on a single modality i.e., text. To bridge this gap, we construct MR2, a multimodal multilingual retrieval-augmented dataset for rumor detection. The dataset covers rumors with images and texts, and provides evidence from both modalities that are retrieved from the Internet. Further, we develop established baselines and conduct a detailed analysis of the systems evaluated on the dataset. Extensive experiments show that MR2 will provide a challenging testbed for developing rumor detection systems designed to retrieve and reason over social media posts. Source code and data are available at: https://github.com/THU-BPM/MR2.
Trustworthy Machine Learning: Robustness, Generalization, and Interpretability
Authors: Jindong Wang 0001, Haoliang Li, Haohan Wang, Sinno Jialin Pan, Xing Xie 0001
Venue: Conference KDD 2023
Year: 2023

Machine learning is becoming increasingly important in today's world. Beyond its powerful performances, there has been an emerging concern about the trustworthiness of machine learning, including but not limited to: robustness to malicious attacks, generalization to unseen datasets, and interpretability to explain its outputs. Such concerns are even more urgent in some safety-critical applications such as medical diagnosis and autonomous driving. Trustworthy machine learning (TrustML) aims to tackle these challenges from the perspectives of theory, algorithm, and applications. In this tutorial, we will give a comprehensive introduction to the recent advance of trustworthy machine learning in robustness, generalization, and interpretability. We will cover their problem formulation, related research, popular algorithms, and successful applications. Additionally, we will also introduce some potential challenges for future research. We do hope that this tutorial will not only serve as a platform to understand TrustML, but also raise the awareness of everyone for more trustworthy applications.
Attentional graph convolutional networks for knowledge concept recommendation in moocs in a heterogeneous view //
Authors: Jibing Gong, Shen Wang 0005, Jinlong Wang 0005, Wenzheng Feng, Hao Peng 0001, Jie Tang 0001, Philip S. Yu
Venue: Conference SIGIR 2020
Year: 2020

Massive open online courses (MOOCs) are becoming a modish way for education, which provides a large-scale and open-access learning opportunity for students to grasp the knowledge. To attract students' interest, the recommendation system is applied by MOOCs providers to recommend courses to students. However, as a course usually consists of a number of video lectures, with each one covering some specific knowledge concepts, directly recommending courses overlook students' interest to some specific knowledge concepts. To fill this gap, in this paper, we study the problem of knowledge concept recommendation. We propose an end-to-end graph neural network based approach calledAttentional Heterogeneous Graph Convolutional Deep Knowledge Recommender (ACKRec) for knowledge concept recommendation in MOOCs. Like other recommendation problems, it suffers from sparsity issue. To address this issue, we leverage both content information and context information to learn the representation of entities via graph convolution network. In addition to students and knowledge concepts, we consider other types of entities (e.g., courses, videos, teachers) and construct a heterogeneous information network (HIN) to capture the corresponding fruitful semantic relationships among different types of entities and incorporate them into the representation learning process. Specifically, we use meta-path on the HIN to guide the propagation of students' preferences. With the help of these meta-paths, the students' preference distribution with respect to a candidate knowledge concept can be captured. Furthermore, we propose an attention mechanism to adaptively fuse the context information from different meta-paths, in order to capture the different interests of different students. To learn the parameters of the proposed model, we propose to utilize extended matrix factorization (MF). A series of experiments are conducted, demonstrating the effectiveness of ACKRec across multiple popular metrics compared with state-of-the-art baseline methods. The promising results show that the proposed ACKRec is able to effectively recommend knowledge concepts to students pursuing online learning in MOOCs.
Cloudrca: A root cause analysis framework for cloud computing platforms
Authors: Yingying Zhang, Zhengxiong Guan, Huajie Qian, Leili Xu, Hengbo Liu, Qingsong Wen, Liang Sun, Junwei Jiang, Lunting Fan, Min Ke
Venue: Conference CIKM 2021
Year: 2021

As business of Alibaba expands across the world among various industries, higher standards are imposed on the service quality and reliability of big data cloud computing platforms which constitute the infrastructure of Alibaba Cloud. However, root cause analysis in these platforms is non-trivial due to the complicated system architecture. In this paper, we propose a root cause analysis framework called CloudRCA which makes use of heterogeneous multi-source data including Key Performance Indicators (KPIs), logs, as well as topology, and extracts important features via state-of-the-art anomaly detection and log analysis techniques. The engineered features are then utilized in a Knowledge-informed Hierarchical Bayesian Network (KHBN) model to infer root causes with high accuracy and efficiency. Ablation study and comprehensive experimental comparisons demonstrate that, compared to existing frameworks, CloudRCA 1) consistently outperforms existing approaches in f1-score across different cloud systems; 2) can handle novel types of root causes thanks to the hierarchical structure of KHBN; 3) performs more robustly with respect to algorithmic configurations; and 4) scales more favorably in the data and feature sizes. Experiments also show that a cross-platform transfer learning mechanism can be adopted to further improve the accuracy by more than 10\%. CloudRCA has been integrated into the diagnosis system of Alibaba Cloud and employed in three typical cloud computing platforms including MaxCompute, Realtime Compute and Hologres. It saves Site Reliability Engineers (SREs) more than $20\%$ in the time spent on resolving failures in the past twelve months and improves service reliability significantly.
Context Attentive Document Ranking and Query Suggestion
Authors: Wasi Uddin Ahmad, Kai-Wei Chang, Hongning Wang
Venue: Conference SIGIR 2019
Year: 2019

We present a context-aware neural ranking model to exploit users' on-task search activities and enhance retrieval performance. In particular, a two-level hierarchical recurrent neural network is introduced to learn search context representation of individual queries, search tasks, and corresponding dependency structure by jointly optimizing two companion retrieval tasks: document ranking and query suggestion. To identify the variable dependency structure between search context and users' ongoing search activities, attention at both levels of recurrent states are introduced. Extensive experiment comparisons against a rich set of baseline methods and an in-depth ablation analysis confirm the value of our proposed approach for modeling search context buried in search tasks.
Generating query suggestions to support task-based search
Authors: Darío Garigliotti, Krisztian Balog
Venue: Conference SIGIR 2017
Year: 2017

We address the problem of generating query suggestions to support users in completing their underlying tasks (which motivated them to search in the first place). Given an initial query, these query suggestions should provide a coverage of possible subtasks the user might be looking for. We propose a probabilistic modeling framework that obtains keyphrases from multiple sources and generates query suggestions from these keyphrases. Using the test suites of the TREC Tasks track, we evaluate and analyze each component of our model.
Personalized Query Suggestions
Authors: Jianling Zhong, Weiwei Guo, Huiji Gao, Bo Long
Venue: Conference SIGIR 2020
Year: 2020

With the exponential growth of information on the internet, users have been relying on search engines for finding the precise documents. However, user queries are often short. The inherent ambiguity of short queries imposes great challenges for search engines to understand user intent. Query suggestion is one key technique for search engines to augment user queries so that they can better understand user intent. In the past, query suggestions have been relying on either term-frequency--based methods with little semantic understanding of the query, or word-embedding--based methods with little personalization efforts. Here, we present a sequence-to-sequence-model--based query suggestion framework that is capable of modeling structured, personalized features and unstructured query texts naturally. This capability opens up the opportunity to better understand query semantics and user intent at the same time. As the largest professional network, LinkedIn has the advantage of utilizing a rich amount of accurate member profile information to personalize query suggestions. We applied this framework in the LinkedIn production traffic and showed that personalized query suggestions significantly improved member search experience as measured by key business metrics at LinkedIn.
Personalized Query Suggestion with Searching Dynamic Flow for Online Recruitment
Authors: Zile Zhou, Xiao Zhou, Mingzhe Li, Yang Song, Tao Zhang, Rui Yan
Venue: Conference CIKM 2022
Year: 2022

Employing query suggestion techniques to assist users in articulating their needs during online search has become increasingly vital for search engines in an age of exponential information growth. The success of a query suggestion system lies in understanding and modeling user search intent behind each query accurately, which can hardly be achieved without personalization efforts on taking advantage of dynamic user feedback behaviors and rich contextual information. This valuable area, however, has been still largely untapped by current query suggestion systems. In this work, we propose Dynamic Searching Flow Model (DSFM), a query suggestion framework that is capable of modeling and refining user search intent progressively in recruitment scenarios by leveraging a dynamic flow mechanism. Here the concepts of local flow and global flow are introduced to capture the real-time intention of users and the overall influence of a session, respectively. By utilizing rich semantic information contained in resumes and job requirements, DSFM enables the personalization of query suggestions. In addition, weighted contrast learning is introduced into the training process to produce more extensive targeted query samples and partially alleviate the exposure bias. The adoption of attention mechanism allows the selection of the most relevant information to compose the final intention representation. Extensive experimental results on different categories of real-world datasets demonstrate the effectiveness of our proposed approach on the task of query suggestion for online recruitment platforms.
Breaking Filter Bubble: A Reinforcement Learning Framework of Controllable Recommender System
Authors: Zhenyang Li, Yancheng Dong, Chen Gao, Yizhou Zhao, Dong Li, Jianye Hao, Kai Zhang, Yong Li, Zhi Wang
Venue: Conference WWW 2023
Year: 2023

In the information-overloaded era of the Web, recommender systems that provide personalized content filtering are now the mainstream portal for users to access Web information. Recommender systems deploy machine learning models to learn users’ preferences from collected historical data, leading to more centralized recommendation results due to the feedback loop. As a result, it will harm the ranking of content outside the narrowed scope and limit the options seen by users. In this work, we first conduct data analysis from a graph view to observe that the users’ feedback is restricted to limited items, verifying the phenomenon of centralized recommendation. We further develop a general simulation framework to derive the procedure of the recommender system, including data collection, model learning, and item exposure, which forms a loop. To address the filter bubble issue under the feedback loop, we then propose a general and easy-to-use reinforcement learning-based method, which can adaptively select few but effective connections between nodes from different communities as the exposure list. We conduct extensive experiments in the simulation framework based on large-scale real-world datasets. The results demonstrate that our proposed reinforcement learning-based control method can serve as an effective solution to alleviate the filter bubble and the separated communities induced by it. We believe the proposed framework of controllable recommendation in this work can inspire not only the researchers of recommender systems, but also a broader community concerned with artificial intelligence algorithms’ impact on humanity, especially for those vulnerable populations on the Web.
A Century of Science: Globalization of Scientific Collaborations, Citations, and Innovations
Authors: Yuxiao Dong, Hao Ma, Zhihong Shen, Kuansan Wang
Venue: Conference KDD 2017
Year: 2017

Progress in science has advanced the development of human society across history, with dramatic revolutions shaped by information theory, genetic cloning, and artificial intelligence, among the many scientific achievements produced in the 20th century. However, the way that science advances itself is much less well-understood. In this work, we study the evolution of scientific development over the past century by presenting an anatomy of 89 million digitalized papers published between 1900 and 2015. We find that science has benefited from the shift from individual work to collaborative effort, with over 90% of the world-leading innovations generated by collaborations in this century, nearly four times higher than they were in the 1900s. We discover that rather than the frequent myopic- and self-referencing that was common in the early 20th century, modern scientists instead tend to look for literature further back and farther around. Finally, we also observe the globalization of scientific development from 1900 to 2015, including 25-fold and 7-fold increases in international collaborations and citations, respectively, as well as a dramatic decline in the dominant accumulation of citations by the US, the UK, and Germany, from ~95% to ~50% over the same period. Our discoveries are meant to serve as a starter for exploring the visionary ways in which science has developed throughout the past century, generating insight into and an impact upon the current scientific innovations and funding policies.
OAG-BERT: Towards a Unified Backbone Language Model for Academic Knowledge Services
Authors: Xiao Liu 0036, Da Yin, Jingnan Zheng, Xingjian Zhang 0009, Peng Zhang 0077, Hongxia Yang, Yuxiao Dong, Jie Tang 0001
Venue: Conference KDD 2022
Year: 2022

Academic Knowledge Services have substantially facilitated the development of human science and technology, providing a plenitude of useful research tools. However, many applications highly depend on ad-hoc models and expensive human labeling to understand professional contents, hindering deployments in real world. To create a unified backbone language model for various knowledge-intensive academic knowledge mining challenges, based on the world's largest public academic graph Open Academic Graph (OAG), we pre-train an academic language model, namely OAG-BERT, to integrate massive heterogeneous entity knowledge beyond scientific corpora. We develop novel pre-training strategies along with zero-shot inference techniques. OAG-BERT's superior performance on 9 knowledge-intensive academic tasks (including 2 demo applications) demonstrates its qualification to serve as a foundation for academic knowledge services. Its zero-shot capability also offers great potential to mitigate the need of costly annotations. OAG-BERT has been deployed to multiple real-world applications, such as reviewer recommendations for NSFC (National Nature Science Foundation of China) and paper tagging in the AMiner system. All codes and pre-trained models are available via the CogDL.
SVMpAUCtight: A new support vector method for optimizing partial AUC based on a tight convex upper bound
Authors: Harikrishna Narasimhan, Shivani Agarwal
Venue: Conference KDD 2013
Year: 2013

The area under the ROC curve (AUC) is a well known performance measure in machine learning and data mining. In an increasing number of applications, however, ranging from ranking applications to a variety of important bioinformatics applications, performance is measured in terms of the partial area under the ROC curve between two specified false positive rates. In recent work, we proposed a structural SVM based approach for optimizing this performance measure (Narasimhan and Agarwal, 2013). In this paper, we develop a new support vector method, SVMpAUCtight, that optimizes a tighter convex upper bound on the partial AUC loss, which leads to both improved accuracy and reduced computational complexity. In particular, by rewriting the empirical partial AUC risk as a maximum over subsets of negative instances, we derive a new formulation, where a modified form of the earlier optimization objective is evaluated on each of these subsets, leading to a tighter hinge relaxation on the partial AUC loss. As with our previous method, the resulting optimization problem can be solved using a cutting-plane algorithm, but the new method has better run time guarantees. We also discuss a projected subgradient method for solving this problem, which offers additional computational savings in certain settings. We demonstrate on a wide variety of bioinformatics tasks, ranging from protein-protein interaction prediction to drug discovery tasks, that the proposed method does, in many cases, perform significantly better on the partial AUC measure than the previous structural SVM approach. In addition, we also develop extensions of our method to learn sparse and group sparse models, often of interest in biological applications.
FedEdge: Accelerating Edge-Assisted Federated Learning
Authors: Kaibin Wang, Qiang He, Feifei Chen, Hai Jin, Yun Yang
Venue: Conference WWW 2023
Year: 2023

Federated learning (FL) has been widely acknowledged as a promising solution to training machine learning (ML) model training with privacy preservation. To reduce the traffic overheads incurred by FL systems, edge servers have been included between clients and the parameter server to aggregate clients’ local models. Recent studies on this edge-assisted hierarchical FL scheme have focused on ensuring or accelerating model convergence by coping with various factors, e.g., uncertain network conditions, unreliable clients, heterogeneous compute resources, etc. This paper presents our three new discoveries of the edge-assisted hierarchical FL scheme: 1) it wastes significant time during its two-phase training rounds; 2) it does not recognize or utilize model diversity when producing a global model; and 3) it is vulnerable to model poisoning attacks. To overcome these drawbacks, we propose FedEdge, a novel edge-assisted hierarchical FL scheme that accelerates model training with asynchronous local federated training and adaptive model aggregation. Extensive experiments are conducted on two widely-used public datasets. The results demonstrate that, compared with state-of-the-art FL schemes, FedEdge accelerates model convergence by 1.14 × −3.20 ×, and improves model accuracy by 2.14% - 6.63%.