OpenRaiser/Intern-Atlas huggingface dataset:
CoopEdge: A decentralized blockchain-based platform for cooperative edge computing
Authors: Liang Yuan, Qiang He, Siyu Tan, Bo Li, Jiangshan Yu, Feifei Chen, Hai Jin, Yun Yang
Venue: Conference WWW 2021
Year: 2021

Edge computing (EC) has recently emerged as a novel computing paradigm that offers users low-latency services. Suffering from constrained computing resources due to their limited physical sizes, edge servers cannot always handle all the incoming computation tasks timely when they operate independently. They often need to cooperate through peer-offloading. Deployed and managed by different stakeholders, edge servers operate in a distrusted environment. Trust and incentive are the two main issues that challenge cooperative computing between them. Another unique challenge in the EC environment is to facilitate trust and incentive in a decentralized manner. To tackle these challenges systematically, this paper proposes CoopEdge, a novel blockchain-based decentralized platform, to drive and support cooperative edge computing. On CoopEdge, an edge server can publish a computation task for other edge servers to contend for. A winner is selected from candidate edge servers based on their reputations. After that, a consensus is reached among edge servers to record the performance in task execution on blockchain. We implement CoopEdge based on Hyperledger Sawtooth and evaluate it experimentally against a baseline and two state-of-the-art implementations in a simulated EC environment. The results validate the usefulness of CoopEdge and demonstrate its performance.
MVAN: Multi-view attention networks for real money trading detection in online games
Authors: Jianrong Tao, Jianshi Lin, Shize Zhang, Sha Zhao, Runze Wu, Changjie Fan, Peng Cui
Venue: Conference KDD 2019
Year: 2019

Online gaming is a multi-billion dollar industry that entertains a large, global population. However, one unfortunate phenomenon known as real money trading harms the competition and the fun. Real money trading is an interesting economic activity used to exchange assets in a virtual world with real world currencies, leading to imbalance of game economy and inequality of wealth and opportunity. Game operation teams have been devoting much efforts on real money trading detection, however, it still remains a challenging task. To overcome the limitation from traditional methods conducted by game operation teams, we propose, MVAN, the first multi-view attention networks for detecting real money trading with multi-view data sources. We present a multi-graph attention network (MGAT) in the graph structure view, a behavior attention network (BAN) in the vertex content view, a portrait attention network (PAN) in the vertex attribute view and a data source attention network (DSAN) in the data source view. Experiments conducted on real-world game logs from a commercial NetEase MMORPG( JusticePC) show that our method consistently performs promising results compared with other competitive methods over time and verifiy the importance and rationality of attention mechanisms. MVAN is deployed to several MMORPGs in NetEase in practice and achieving remarkable performance improvement and acceleration. Our method can easily generalize to other types of related tasks in real world, such as fraud detection, drug tracking and money laundering tracking etc.
How to retrain recommender system? a sequential meta-learning method
Authors: Yang Zhang 0072, Fuli Feng, Chenxu Wang 0010, Xiangnan He 0001, Meng Wang 0001, Yan Li 0068, Yongdong Zhang 0001
Venue: Conference SIGIR 2020
Year: 2020

Practical recommender systems need be periodically retrained to refresh the model with new interaction data. To pursue high model fidelity, it is usually desirable to retrain the model on both historical and new data, since it can account for both long-term and short-term user preference. However, a full model retraining could be very time-consuming and memory-costly, especially when the scale of historical data is large. In this work, we study the model retraining mechanism for recommender systems, a topic of high practical values but has been relatively little explored in the research community. Our first belief is that retraining the model on historical data is unnecessary, since the model has been trained on it before. Nevertheless, normal training on new data only may easily cause overfitting and forgetting issues, since the new data is of a smaller scale and contains fewer information on long-term user preference. To address this dilemma, we propose a new training method, aiming to abandon the historical data during retraining through learning to transfer the past training experience. Specifically, we design a neural network-based transfer component, which transforms the old model to a new model that is tailored for future recommendations. To learn the transfer component well, we optimize the "future performance" -- i.e., the recommendation accuracy evaluated in the next time period. Our Sequential Meta-Learning(SML) method offers a general training paradigm that is applicable to any differentiable model. We demonstrate SML on matrix factorization and conduct experiments on two real-world datasets. Empirical results show that SML not only achieves significant speed-up, but also outperforms the full model retraining in recommendation accuracy, validating the effectiveness of our proposals. We release our codes at: https://github.com/zyang1580/SML.
Federated topic modeling
Authors: Di Jiang, Yuanfeng Song, Yongxin Tong, Xueyang Wu, Weiwei Zhao, Qian Xu, Qiang Yang
Venue: Conference CIKM 2019
Year: 2019

Topic modeling has been widely applied in a variety of industrial applications. Training a high-quality model usually requires massive amount of in-domain data, in order to provide comprehensive co-occurrence information for the model to learn. However, industrial data such as medical or financial records are often proprietary or sensitive, which precludes uploading to data centers. Hence training topic models in industrial scenarios using conventional approaches faces a dilemma: a party (i.e., a company or institute) has to either tolerate data scarcity or sacrifice data privacy. In this paper, we propose a novel framework named Federated Topic Modeling (FTM), in which multiple parties collaboratively train a high-quality topic model by simultaneously alleviating data scarcity and maintaining immune to privacy adversaries. FTM is inspired by federated learning and consists of novel techniques such as private Metropolis Hastings, topic-wise normalization and heterogeneous model integration. We conduct a series of quantitative evaluations to verify the effectiveness of FTM and deploy FTM in an Automatic Speech Recognition (ASR) system to demonstrate its utility in real-life applications. Experimental results verify FTM's superiority over conventional topic modeling.
Count-min: Optimal estimation and tight error bounds using empirical error distributions
Authors: Daniel Ting
Venue: Conference KDD 2018
Year: 2018

The Count-Min sketch is an important and well-studied data summarization method. It allows one to estimate the count of any item in a stream using a small, fixed size data sketch. However, the accuracy of the sketch depends on characteristics of the underlying data. This has led to a number of count estimation procedures which work well in one scenario but perform poorly in others. A practitioner is faced with two basic, unanswered questions. Which variant should be chosen when the data is unknown? Given an estimate, is its error sufficiently small to be trustworthy? We provide answers to these questions. We derive new count estimators, including a provably optimal estimator, which best or match previous estimators in all scenarios. We also provide practical, tight error bounds at query time for both new and existing estimators. These error estimates also yield procedures to choose the sketch tuning parameters optimally, as they can extrapolate the error to different choices of sketch width and depth. The key observation is that the distribution of errors in each counter can be empirically estimated from the sketch itself. By first estimating this distribution, count estimation becomes a statistical estimation and inference problem with a known error distribution. This provides both a principled way to derive new and optimal estimators as well as a way to study the error and properties of existing estimators.
On the discovery of evolving truth
Authors: Yaliang Li, Qi Li, Jing Gao, Lu Su, Bo Zhao, Wei Fan, Jiawei Han
Venue: Conference KDD 2015
Year: 2015

In the era of big data, information regarding the same objects can be collected from increasingly more sources. Unfortunately, there usually exist conflicts among the information coming from different sources. To tackle this challenge, truth discovery, i.e., to integrate multi-source noisy information by estimating the reliability of each source, has emerged as a hot topic. In many real world applications, however, the information may come sequentially, and as a consequence, the truth of objects as well as the reliability of sources may be dynamically evolving. Existing truth discovery methods, unfortunately, cannot handle such scenarios. To address this problem, we investigate the temporal relations among both object truths and source reliability, and propose an incremental truth discovery framework that can dynamically update object truths and source weights upon the arrival of new data. Theoretical analysis is provided to show that the proposed method is guaranteed to converge at a fast rate. The experiments on three real world applications and a set of synthetic data demonstrate the advantages of the proposed method over state-of-the-art truth discovery methods.
AdaRNN: Adaptive learning and forecasting of time series
Authors: Yuntao Du, Jindong Wang, Wenjie Feng, Sinno Jialin Pan, Tao Qin, Renjun Xu, Chongjun Wang
Venue: Conference CIKM 2021
Year: 2021

Time series has wide applications in the real world and is known to be difficult to forecast. Since its statistical properties change over time, its distribution also changes temporally, which will cause severe distribution shift problem to existing methods. However, it remains unexplored to model the time series in the distribution perspective. In this paper, we term this as Temporal Covariate Shift (TCS). This paper proposes Adaptive RNNs (AdaRNN) to tackle the TCS problem by building an adaptive model that generalizes well on the unseen test data. AdaRNN is sequentially composed of two novel algorithms. First, we propose Temporal Distribution Characterization to better characterize the distribution information in the TS. Second, we propose Temporal Distribution Matching to reduce the distribution mismatch in TS to learn the adaptive TS model. AdaRNN is a general framework with flexible distribution distances integrated. Experiments on human activity recognition, air quality prediction, and financial analysis show that AdaRNN outperforms the latest methods by a classification accuracy of 2.6% and significantly reduces the RMSE by 9.0%. We also show that the temporal distribution matching algorithm can be extended in Transformer structure to boost its performance.
Query-based interactive recommendation by meta-path and adapted attention-gru
Authors: Yu Zhu, Yu Gong, Qingwen Liu, Yingcai Ma, Wenwu Ou, Junxiong Zhu, Beidou Wang, Ziyu Guan, Deng Cai
Venue: Conference CIKM 2019
Year: 2019

Recently, interactive recommender systems are becoming increasingly popular. The insight is that, with the interaction between users and the system, (1) users can actively intervene the recommendation results rather than passively receive them, and (2) the system learns more about users so as to provide better recommendation. We focus on the single-round interaction, i.e. the system asks the user a question (Step 1), and exploits his feedback to generate better recommendation (Step 2). A novel query-based interactive recommender system is proposed in this paper, where \textbf{personalized questions are accurately generated from millions of automatically constructed questions} in Step 1, and \textbf{the recommendation is ensured to be closely-related to users' feedback} in Step 2. We achieve this by transforming Step 1 into a query recommendation task and Step 2 into a retrieval task. The former task is our key challenge. We firstly propose a model based on Meta-Path to efficiently retrieve hundreds of query candidates from the large query pool. Then an adapted Attention-GRU model is developed to effectively rank these candidates for recommendation. Offline and online experiments on Taobao, a large-scale e-commerce platform in China, verify the effectiveness of our interactive system. The system has already gone into production in the homepage of Taobao App since Nov. 11, 2018 (see https://v.qq.com/x/page/s0833tkp1uo.html on how it works online). Our code and dataset are public in https://github.com/zyody/QueryQR.
Hyperbolic hypergraphs for sequential recommendation
Authors: Yicong Li, Hongxu Chen, Xiangguo Sun, Zhenchao Sun, Lin Li, Lizhen Cui, Philip S. Yu, Guandong Xu
Venue: Conference CIKM 2021
Year: 2021

Hypergraphs have been becoming a popular choice to model complex, non-pairwise, and higher-order interactions for recommender systems. However, compared with traditional graph-based methods, the constructed hypergraphs are usually much sparser, which leads to a dilemma when balancing the benefits of hypergraphs and the modelling difficulty. Moreover, existing sequential hypergraph recommendation overlooks the temporal modelling among user relationships, which neglects rich social signals from the recommendation data. To tackle the above shortcomings of the existing hypergraph-based sequential recommendations, we propose a novel architecture named Hyperbolic Hypergraph representation learning method for Sequential Recommendation (H2SeqRec) with the pre-training phase. Specifically, we design three self-supervised tasks to obtain the pre-training item embeddings to feed or fuse into the following recommendation architecture (with two ways to use the pre-trained embeddings). In the recommendation phase, we learn multi-scale item embeddings via a hierarchical structure to capture multiple time-span information. To alleviate the negative impact of sparse hypergraphs, we utilize a hyperbolic space-based hypergraph convolutional neural network to learn the dynamic item embeddings. Also, we design an item enhancement module to capture dynamic social information at each timestamp to improve effectiveness. Extensive experiments are conducted on two real-world datasets to prove the effectiveness and high performance of the model.
Causalint: Causal inspired intervention for multi-scenario recommendation
Authors: Yichao Wang 0002, Huifeng Guo, Bo Chen 0023, Weiwen Liu, Zhirong Liu, Qi Zhang 0001, Zhicheng He 0001, Hongkun Zheng, Weiwei Yao, Muyu Zhang, Zhenhua Dong, Ruiming Tang
Venue: Conference KDD 2022
Year: 2022

Building appropriate scenarios to meet the personalized demands of different user groups is a common practice. Despite various scenario brings personalized service, it also leads to challenges for the recommendation on multiple scenarios, especially the scenarios with limited traffic. To give desirable recommendation service for all scenarios and reduce the cost of resource consumption, how to leverage the information from multiple scenarios to construct a unified model becomes critical. Unfortunately, the performance of existing multi-scenario recommendation approaches is poor since they introduce unnecessary information from other scenarios to target scenario. In this paper, we show it is possible to selectively utilize the information from different scenarios to construct the scenario-aware estimators in a unified model. Specifically, we first do analysis on multi-scenario modeling with causal graph from the perspective of users and modeling processes, and then propose the Causal Inspired Intervention (CausalInt) framework for multi-scenario recommendation. CausalInt consists of three modules: (1) Invariant Representation Modeling module to squeeze out the scenario-aware information through disentangled representation learning and obtain a scenario-invariant representation; (2) Negative Effects Mitigating module to resolve conflicts between different scenarios and conflicts between scenario-specific and scenario-invariant representations via gradient based orthogonal regularization and model-agnostic meta learning, respectively; (3) Inter-Scenario Transferring module designs a novel TransNet to simulate a counterfactual intervention and effectively fuse the information from other scenarios. Offline experiments over two real-world dataset and online A/B test are conducted to demonstrate the superiority of CausalInt.
Ensembled ctr prediction via knowledge distillation
Authors: Jieming Zhu, Jinyang Liu, Weiqi Li, Jincai Lai, Xiuqiang He, Liang Chen, Zibin Zheng
Venue: Conference CIKM 2020
Year: 2020

Recently, deep learning-based models have been widely studied for click-through rate (CTR) prediction and lead to improved prediction accuracy in many industrial applications. However, current research focuses primarily on building complex network architectures to better capture sophisticated feature interactions and dynamic user behaviors. The increased model complexity may slow down online inference and hinder its adoption in real-time applications. Instead, our work targets at a new model training strategy based on knowledge distillation (KD). KD is a teacher-student learning framework to transfer knowledge learned from a teacher model to a student model. The KD strategy not only allows us to simplify the student model as a vanilla DNN model but also achieves significant accuracy improvements over the state-of-the-art teacher models. The benefits thus motivate us to further explore the use of a powerful ensemble of teachers for more accurate student model training. We also propose some novel techniques to facilitate ensembled CTR prediction, including teacher gating and early stopping by distillation loss. We conduct comprehensive experiments against 12 existing models and across three industrial datasets. Both offline and online A/B testing results show the effectiveness of our KD-based training strategy.
Improving relevance modeling via heterogeneous behavior graph learning in bing ads
Authors: Bochen Pang, Chaozhuo Li, Yuming Liu, Jianxun Lian, Jianan Zhao 0002, Hao Sun 0015, Weiwei Deng, Xing Xie 0001, Qi Zhang 0066
Venue: Conference KDD 2022
Year: 2022

As the fundamental basis of sponsored search, relevance modeling measures the closeness between the input queries and the candidate ads. Conventional relevance models solely rely on the textual data, which suffer from the scarce semantic signals within the short queries. Recently, user historical click behaviors are incorporated in the format of click graphs to provide additional correlations beyond pure textual semantics, which contributes to advancing the relevance modeling performance. However, user behaviors are usually arbitrary and unpredictable, leading to the noisy and sparse graph topology. In addition, there exist other types of user behaviors besides clicks, which may also provide complementary information. In this paper, we study the novel problem of heterogeneous behavior graph learning to facilitate relevance modeling task. Our motivation lies in learning an optimal and task-relevant heterogeneous behavior graph consisting of multiple types of user behaviors. We further propose a novel HBGLR model to learn the behavior graph structure by mining the sophisticated correlations between node semantics and graph topology, and encode the textual semantics and structural heterogeneity into the learned representations. Our proposal is evaluated over real-world industry datasets, and has been mainstreamed in the Bing ads. Both offline and online experimental results demonstrate its superiority.
Beyond user embedding matrix: Learning to hash for modeling large-scale users in recommendation
Authors: Shaoyun Shi, Weizhi Ma, Min Zhang 0006, Yongfeng Zhang 0003, Xinxing Yu, Houzhi Shan, Yiqun Liu 0001, Shaoping Ma
Venue: Conference SIGIR 2020
Year: 2020

Modeling large scale and rare-interaction users are the two major challenges in recommender systems, which derives big gaps between researches and applications. Facing to millions or even billions of users, it is hard to store and leverage personalized preferences with a user embedding matrix in real scenarios. And many researches pay attention to users with rich histories, while users with only one or several interactions are the biggest part in real systems. Previous studies make efforts to handle one of the above issues but rarely tackle efficiency and cold-start problems together.
AMiner: Toward understanding big scholar data
Authors: Jie Tang
Venue: Conference WSDM 2016
Year: 2016

In this talk, I will present a novel academic search and mining system, AMiner, the second generation of the ArnetMiner system. Different from traditional academic search systems that focus on document (paper) search, AMiner aims to provide a systematic modeling approach for researchers (authors), ultimately to gain a deep understanding of the big (heterogeneous) network formed by authors, papers they have published, and venues they published those papers. The system extracts researchers' profiles automatically from the Web and integrates the researcher profiles with publication papers after name disambiguation. For now, the system has collected a big scholar data with more than 130,000,000 researcher profiles and 100,000,000 papers from multiple publication databases. We also developed an approach named COSNET to connect AMiner with several professional social networks such as LinkedIn and VideoLectures, which significantly enriches the metadata of the scholarly data. Based on the integrated big scholar data, we devise a unified topic modeling approach for modeling the different entities (authors, papers, venues) simultaneously and provide a topic-level expertise search by leveraging the modeling results. In addition, AMiner offers a set of researcher-centered functions including social influence analysis, influence visualization, collaboration recommendation, relationship mining, similarity analysis and community evolution. The system has been put into operation since 2006 and has attracted more than 7,000,000 independent IP accesses from over 200 countries/regions.