OpenRaiser/Intern-Atlas huggingface dataset:
Obtaining Robust Models from Imbalanced Data
Authors: Wentao Wang
Venue: WSDM
Year: 2022

The vulnerability of deep neural network (DNN) models has been verified by the existence of adversarial examples. By exploiting slight changes to input examples, the generated adversarial examples can easily cause well trained DNN models make wrong predictions. Many defense methods have been proposed to improve the robustness of DNN models against adversarial examples. Among them, adversarial training has been empirically proven to be one of the most effective methods. Almost all existing studies about adversarial training are focused on balanced datasets, where each class has an equal amount of training examples. However, as datasets collected in real-world applications cannot guarantee all contained classes are uniformly distributed, it would be much challenging to obtain robust models in those real applications where the available training datasets are imbalanced. As the initial effort to study this problem, we first investigate the different behaviors between adversarially trained models and naturally trained models using imbalanced training datasets and then explore possible solutions to facilitate adversarial training under imbalanced settings.
Rethink e-Commerce Search
Authors: Haixun Wang
Venue: WSDM
Year: 2022

The quality of the search experience on an e-commerce site plays a critical role in customer conversion and the growth of the e-commerce business. In this talk, I will discuss the current status and challenges of product search. In particular, I will highlight the significant amount of effort it takes to create a high-quality product search engine using classical information retrieval methods. Then, I will discuss how recent advances in NLP and deep learning, especially the advent of large pre-trained language models, may change the status quo. While embedding-based retrieval has the potential to improve classical information retrieval methods, creating a machine learning-based, end-to-end system for general-purpose, web search is still extremely difficult. Nevertheless, I will argue that product search for e-commerce may prove to be an area where deep learning can create the first disruption to classical information retrieval systems.
Learning-To-Ensemble by Contextual Rank Aggregation in E-Commerce
Authors: Xuesi Wang, Guangda Huzhang, Qianying Lin, Qing Da
Venue: WSDM
Year: 2022

Ensemble models in E-commerce combine predictions from multiple sub-models for ranking and revenue improvement. Industrial ensemble models are typically deep neural networks, following the supervised learning paradigm to infer conversion rate given inputs from sub-models. However, this process has the following two problems. Firstly, the point-wise scoring approach disregards the relationships between items and leads to homogeneous displayed results, while diversified display benefits user experience and revenue. Secondly, the learning paradigm focuses on the ranking metrics and does not directly optimize the revenue. In our work, we propose a new Learning-To-Ensemble (LTE) framework RA-EGO, which replaces the ensemble model with a contextual Rank Aggregator (RA) and explores the best weights of sub-models by the Evaluator-Generator Optimization (EGO). To achieve the best online performance, we propose a new rank aggregation algorithm TournamentGreedy as a refinement of classic rank aggregators, which also produces the best average weighted Kendall Tau Distance (KTD) amongst all the considered algorithms with quadratic time complexity. Under the assumption that the best output list should be Pareto Optimal on the KTD metric for sub-models, we show that our RA algorithm has higher efficiency and coverage in exploring the optimal weights. Combined with the idea of Bayesian Optimization and gradient descent, we solve the online contextual Black-Box Optimization task that finds the optimal weights for sub-models given a chosen RA model. RA-EGO has been deployed in our online system and has improved the revenue significantly.
A Sequence-to-Sequence Model for Large-scale Chinese Abbreviation Database Construction
Authors: Chao Wang, Jingping Liu, Tianyi Zhuang, Jiahang Li, Juntao Liu, Yanghua Xiao, Wei Wang, Rui Xie
Venue: WSDM
Year: 2022

Abbreviations often used in our daily communication play an important role in natural language processing. Most of the existing studies regard the Chinese abbreviation prediction as a sequence labeling problem. However, sequence labeling models usually ignore label dependencies in the process of abbreviation prediction, and the label prediction of each character should be conditioned on its previous labels. In this paper, we propose to formalize the Chinese abbreviation prediction task as a sequence generation problem, and a novel sequence-to-sequence model is designed. To boost the performance of our deep model, we further propose a multi-level pre-trained model that incorporates character, word, and concept-level embeddings. To evaluate our methods, a new dataset for Chinese abbreviation prediction is automatically built, which contains 81,351 pairs of full forms and abbreviations. Finally, we conduct extensive experiments on a public dataset and the built dataset, and the experimental results on both datasets show that our model outperforms the state-of-the-art methods. More importantly, we build a large-scale database for a specific domain, i.e., life services in Meituan Inc., with high accuracy of about 82.7%, which contains 4,134,142 pairs of full forms and abbreviations. The online A/B testing on Meituan APP and Dianping APP suggests that Click-Through Rate increases by 0.59% and 0.86% respectively when the built database is used in the searching system. We have released our API on http://kw.fudan.edu.cn/ddemos/abbr/ with over 87k API calls in 9 months.
MtCut: A Multi-Task Framework for Ranked List Truncation
Authors: Dong Wang, Jianxin Li, Tianchen Zhu, Haoyi Zhou, Qishan Zhu, Yuxin Wen, Hongming Piao
Venue: WSDM
Year: 2022

Ranked list truncation aims to cut the ranked results in short considering user-defined objectives, which balances the overall utility and user efforts over retrieval results. The exact selection of an optimal cut-off position brings potential benefits in various real-world applications, such as patent search and legal search. However, there is significant retrieval bias in the ranked list. The result scores and the disorder of document sequences cause difficulties in judging the relevance between the queries and documents -- alleviating the existing methods' performance improvement. In this work, we investigate the characteristics of retrieval bias on altering truncation and propose a multi-task truncation model, MtCut. It employs two auxiliary tasks to make complementary for the retrieval bias. As a practical evaluation, we explore its performance on two datasets, and the results show that MtCut outperforms the state-of-the-art methods on both F1-score and DCG metrics.
Structure Meets Sequences: Predicting Network of Co-evolving Sequences
Authors: Yaojing Wang, Yuan Yao, Feng Xu, Yada Zhu, Hanghang Tong
Venue: WSDM
Year: 2022

Co-evolving sequences are ubiquitous in a variety of applications, where different sequences are often inherently inter-connected with each other. We refer to such sequences, together with their inherent connections modeled as a structured network, as network of co-evolving sequences (NoCES). Typical NoCES applications include road traffic monitoring, company revenue prediction, motion capture, etc. To date, it remains a daunting challenge to accurately model NoCES due to the coupling between network structure and sequences. In this paper, we propose to modeling \pname\ with the aim of simultaneously capturing both the dynamics and the interplay between network structure and sequences. Specifically, we propose a joint learning framework to alternatively update the network representations and sequence representations as the sequences evolve over time. A unique feature of our framework lies in that it can deal with the case when there are co-evolving sequences on both network nodes and edges. Experimental evaluations on four real datasets demonstrate that the proposed approach (1) outperforms the existing competitors in terms of prediction accuracy, and (2) scales linearly w.r.t. the sequence length and the network size.
Scalable Graph Topology Learning via Spectral Densification
Authors: Yongyu Wang, Zhiqiang Zhao, Zhuo Feng
Venue: WSDM
Year: 2022

Graph learning plays an important role in many data mining and machine learning tasks, such as manifold learning, data representation and analysis, dimensionality reduction, data clustering, and visualization, etc. In this work, we introduce a highly-scalable spectral graph densification approach (GRASPEL) for graph topology learning from data. By limiting the precision matrix to be a graph-Laplacian-like matrix, our approach aims to learn sparse undirected graphs from potentially high-dimensional input data. A very unique property of the graphs learned by GRASPEL is that the spectral embedding (or approximate effective-resistance) distances on the graph will encode the similarities between the original input data points. By leveraging high-performance spectral methods, sparse yet spectrally-robust graphs can be learned by identifying and including the most spectrally-critical edges into the graph. Compared with prior state-of-the-art graph learning approaches, GRASPEL is more scalable and allows substantially improving computing efficiency and solution quality of a variety of data mining and machine learning applications, such as manifold learning, spectral clustering (SC), and dimensionality reduction (DR).
Profiling the Design Space for Graph Neural Networks based Collaborative Filtering
Authors: Zhenyi Wang, Huan Zhao, Chuan Shi
Venue: WSDM
Year: 2022

In recent years, Graph Neural Networks (GNNs) have been widely used in Collaborative Filtering (CF), one of the most popular methods in recommender systems. However, most existing works focus on designing an individual model architecture given a specific scenario, without studying the influences of different design dimensions. Thus, it remains a challenging problem to quickly obtain a top-performing model in a new recommendation scenario. To address the problem, in this work, we make the first attempt to profile the design space of GNN-based CF methods to enrich the understanding of different design dimensions as well as provide a novel paradigm of model design. Specifically, a unified framework of GNN-based CF is proposed, on top of which a design space is developed and evaluated by extensive experiments. Interesting findings on the impacts of different design dimensions on recommendation performance are obtained. Guided by the empirical findings, we further prune the design space to obtain a compact one containing a higher concentration of top-performing models. Empirical studies demonstrate its high quality and strong generalization ability.
A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising
Authors: Chao Wen, Miao Xu, Zhilin Zhang, Zhenzhe Zheng, Yuhui Wang, Xiangyu Liu, Yu Rong, Dong Xie, Xiaoyang Tan, Chuan Yu, Jian Xu, Fan Wu, Guihai Chen, Xiaoqiang Zhu, Bo Zheng
Venue: WSDM
Year: 2022

In online advertising, auto-bidding has become an essential tool for advertisers to optimize their preferred ad performance metrics by simply expressing high-level campaign objectives and constraints. Previous works designed auto-bidding tools from the view of single-agent, without modeling the mutual influence between agents. In this paper, we instead consider this problem from a distributed multi-agent perspective, and propose a general \underlineM ulti-\underlineA gent reinforcement learning framework for \underlineA uto-\underlineB idding, namely MAAB, to learn the auto-bidding strategies. First, we investigate the competition and cooperation relation among auto-bidding agents, and propose a temperature-regularized credit assignment to establish a mixed cooperative-competitive paradigm. By carefully making a competition and cooperation trade-off among agents, we can reach an equilibrium state that guarantees not only individual advertiser's utility but also the system performance (i.e., social welfare). Second, to avoid the potential collusion behaviors of bidding low prices underlying the cooperation, we further propose bar agents to set a personalized bidding bar for each agent, and then alleviate the revenue degradation due to the cooperation. Third, to deploy MAAB in the large-scale advertising system with millions of advertisers, we propose a mean-field approach. By grouping advertisers with the same objective as a mean auto-bidding agent, the interactions among the large-scale advertisers are greatly simplified, making it practical to train MAAB efficiently. Extensive experiments on the offline industrial dataset and Alibaba advertising platform demonstrate that our approach outperforms several baseline methods in terms of social welfare and revenue.
Crowdsourcing-based Multi-Device Communication Cooperation for Mobile High-Quality Video Enhancement
Authors: Xiaotong Wu, Lianyong Qi, Xiaolong Xu, Shui Yu, Wanchun Dou, Xuyun Zhang
Venue: WSDM
Year: 2022

The widespread use of mobile devices propels the development of new-fashioned video applications like 3D (3-Dimensional) stereo video and mobile cloud game via web or App, exerting more pressure on current mobile access network. To address this challenge, we adopt the crowdsourcing paradigm to offer some incentive for guiding the movement of recruited crowdsourcing users and facilitate the optimization of the movement control decision. In this paper, based on a practical 4G (4th-Generation) network throughput measurement study, we formulate the movement control decision as a cost-constrained user recruitment optimization problem. Considering the intractable complexity of this problem, we focus first on a single crowdsourcing user case and propose a pseudo-polynomial time complexity optimal solution. Then, we apply this solution to solve the more general problem of multiple users and propose a graph-partition-based algorithm. Extensive experiments show that our solutions can improve the efficiency of real-time D2D communication for mobile videos.
Long Short-Term Temporal Meta-learning in Online Recommendation
Authors: Ruobing Xie, Yalong Wang, Rui Wang, Yuanfu Lu, Yuanhang Zou, Feng Xia, Leyu Lin
Venue: WSDM
Year: 2022

An effective online recommendation system should jointly capture users' long-term and short-term preferences in both users' internal behaviors (from the target recommendation task) and external behaviors (from other tasks). However, it is extremely challenging to conduct fast adaptations to real-time new trends while making full use of all historical behaviors in large-scale systems, due to the real-world limitations in real-time training efficiency and external behavior acquisition. To address these practical challenges, we propose a novel Long Short-Term Temporal Meta-learning framework (LSTTM) for online recommendation. It arranges user multi-source behaviors in a global long-term graph and an internal short-term graph, and conducts different GAT-based aggregators and training strategies to learn user short-term and long-term preferences separately. To timely capture users' real-time interests, we propose a temporal meta-learning method based on MAML under an asynchronous optimization strategy for fast adaptation, which regards recommendations at different time periods as different tasks. In experiments, LSTTM achieves significant improvements on both offline and online evaluations. It has been deployed on a widely-used online recommendation system named WeChat Top Stories, affecting millions of users.
A Peep into the Future: Adversarial Future Encoding in Recommendation
Authors: Ruobing Xie, Shaoliang Zhang, Rui Wang, Feng Xia, Leyu Lin
Venue: WSDM
Year: 2022

Personalized recommendation often relies on user historical behaviors to provide items for users. It is intuitive that future information also contains essential messages as supplements to user historical behaviors. However, we cannot directly encode future information into models, since we are unable to get future information in online serving. In this work, we propose a novel adversarial future encoding (AFE) framework to make full use of informative future features in different types of recommendation models. Specifically, AFE contains a future-aware discriminator and a generator. The future-aware discriminator takes both common features and future features as inputs, working as a recommendation prophet to judge user-item pairs. In contrast, the generator is considered as a challenger, which generates items with only common features, aiming to confuse the future-aware prophet. The future-aware discriminator can inspire the generator (to be deployed online) to produce better results. We further conduct a multi-factor optimization to enable a fast and stable model convergence via the direct learning and knowledge distillation losses. Moreover, we have adopted AFE on both a list-wise RL-based ranking model and a point-wise ranking model to verify its universality. In experiments, we conduct sufficient evaluations on two large-scale datasets, achieving significant improvements on both offline and online evaluations. Currently, we have deployed AFE on a real-world system, affecting millions of users. The source code is in https://github.com/modriczhang/AFE.
Doctoral Consortium of WSDM'22: Exploring the Bias of Adversarial Defenses
Authors: Han Xu
Venue: WSDM
Year: 2022

Deep neural networks (DNNs) have achieved extraordinary accomplishments on various machine learning tasks. However, the existence of adversarial attacks still raise great concerns when they are adopted to safety-critical tasks. As countermeasures to protect DNN models against adversarial attacks, there are various defense strategies proposed. However, we find that the robustness ("safety'') provided by the robust training algorithms usually result unequal performance either among classes or sub-populations across the whole data distribution. For example, the model can achieve extremely low accuracy / robustness on certain groups of data. As a result, the safety of the model is still under great threats. As a summary, our project is about to study the bias problems of robust trained neural networks from different perspectives, which aims to build eventually reliable and safe deep learning models. We propose to present our research works in the Doctoral Consortium in WSDM'22 and gain opportunities to share our contribution to the relate problems.
Modern Theoretical Tools for Understanding and Designing Next-generation Information Retrieval System
Authors: Da Xu, Chuanwei Ruan
Venue: WSDM
Year: 2022

In the relatively short history of machine learning, the subtle balance between engineering and theoretical progress has been proved critical at various stages. The most recent wave of AI has brought to the IR community powerful techniques, particularly for pattern recognition. While many benefits from the burst of ideas as numerous tasks become algorithmically feasible, the balance is tilting toward the application side. The existing theoretical tools in IR can no longer explain, guide, and justify the newly-established methodologies. With no choices, we have to bet our design on black-box mechanisms that we only empirically understand. The consequences can be suffering: in stark contrast to how the IR industry has envisioned modern AI making life easier, many are experiencing increased confusion and costs in data manipulation, model selection, monitoring, censoring, and decision making. This reality is not surprising: without handy theoretical tools, we often lack principled knowledge of the pattern recognition model's expressivity, optimization property, generalization guarantee, and our decision-making process has to rely on over-simplified assumptions and human judgments from time to time. Facing all the challenges, we started researching advanced theoretical tools emerging from various domains that can potentially resolve modern IR problems. We encountered many impactful ideas and made several independent publications emphasizing different pieces. Time is now to bring the community a systematic tutorial on how we successfully adapt those tools and make significant progress in understanding, designing, and eventually productionize impactful IR systems. We emphasize systematicity because IR is a comprehensive discipline that touches upon particular aspects of learning, causal inference analysis, interactive (online) decision-making, etc. It thus requires systematic calibrations to render the actual usefulness of the imported theoretical tools to serve IR problems, as they usually exhibit unique structures and definitions. Therefore, we plan this tutorial to systematically demonstrate our learning and successful experience of using advanced theoretical tools for understanding and designing IR systems.
Personalized Information Retrieval for Touristic Attractions in Augmented Reality
Authors: Felix Yang, Saikishore Kalloori, Ribin Chalumattu, Markus Gross
Venue: WSDM
Year: 2022

The rapid advances and increasing accessibility of augmented reality (AR) in recent years opened up many new possibilities to incorporate AR into our daily lives. A very interesting area for AR is tourism where one can enhance attractions with virtual elements and provide tourists with additional information about the places they are visiting. In this paper, we present our prototype, an AR application that augments various points of interest (POIs) by showing images and facts about each POI. We also developed a simple recommender system that ensures the facts are selected based on user preferences, thus creating a unique and personalized experience for each user. Furthermore, we also conducted a live user study to assess the usability of our prototype and the usefulness of our personalization system.
Lightweight Composite Re-Ranking for Efficient Keyword Search with BERT
Authors: Yingrui Yang, Yifan Qiao, Jinjin Shao, Xifeng Yan, Tao Yang
Venue: WSDM
Year: 2022

Recently transformer-based ranking models have been shown to deliver high relevance for document search and the relevance-efficiency tradeoff becomes important for fast query response times. This paper presents BECR (BERT-based Composite Re-Ranking), a lightweight composite re-ranking scheme that combines deep contextual token interactions and traditional lexical term-matching features. BECR conducts query decomposition and composes a query representation using pre-computable token embeddings based on uni-grams and skip-n-grams, to seek a tradeoff of inference efficiency and relevance. Thus it does not perform expensive transformer computations during online inference, and does not require the use of GPU. This paper describes an evaluation of relevance and efficiency of BECR with several TREC datasets.