Reviewerly/exHarmony huggingface dataset:
GROUP: An End-to-end Multi-step-ahead Workload Prediction Approach Focusing on Workload Group Behavior

Accurately forecasting workloads can enable web service providers to achieve proactive runtime management for applications and ensure service quality and cost efficiency. For cloud-native applications, multiple containers collaborate to handle user requests, making each container’s workload changes influenced by workload group behavior. However, existing approaches mainly analyze the individual changes of each container and do not explicitly model the workload group evolution of containers, resulting in sub-optimal results. Therefore, we propose a workload prediction method, GROUP, which implements the shifts of workload prediction focus from individual to group, workload group behavior representation from data similarity to data correlation, and workload group behavior evolution from implicit modeling to explicit modeling. First, we model the workload group behavior and its evolution from multiple perspectives. Second, we propose a container correlation calculation algorithm that considers static and dynamic container information to represent the workload group behavior. Third, we propose an end-to-end multi-step-ahead prediction method that explicitly portrays the complex relationship between the evolution of workload group behavior and the workload changes of each container. Lastly, enough experiments on public datasets show the advantages of GROUP, which provides an effective solution to achieve workload prediction for cloud-native applications.
Fair Graph Representation Learning via Diverse Mixture-of-Experts

Graph Neural Networks (GNNs) have demonstrated a great representation learning capability on graph data and have been utilized in various downstream applications. However, real-world data in web-based applications (e.g., recommendation and advertising) always contains bias, preventing GNNs from learning fair representations. Although many works were proposed to address the fairness issue, they suffer from the significant problem of insufficient learnable knowledge with limited attributes after debiasing. To address this problem, we develop Graph-Fairness Mixture of Experts (G-Fame), a novel plug-and-play method to assist any GNNs to learn distinguishable representations with unbiased attributes. Furthermore, based on G-Fame, we propose G-Fame++, which introduces three novel strategies to improve the representation fairness from node representations, model layer, and parameter redundancy perspectives. In particular, we first present the embedding diversified method to learn distinguishable node representations. Second, we design the layer diversified strategy to maximize the output difference of distinct model layers. Third, we introduce the expert diversified method to minimize expert parameter similarities to learn diverse and complementary representations. Extensive experiments demonstrate the superiority of G-Fame and G-Fame++ in both accuracy and fairness, compared to state-of-the-art methods across multiple graph datasets.
PARROT: Position-Aware Regularized Optimal Transport for Network Alignment

Network alignment is a critical steppingstone behind a variety of multi-network mining tasks. Most of the existing methods essentially optimize a Frobenius-like distance or ranking-based loss, ignoring the underlying geometry of graph data. Optimal transport (OT), together with Wasserstein distance, has emerged to be a powerful approach accounting for the underlying geometry explicitly. Promising as it might be, the state-of-the-art OT-based alignment methods suffer from two fundamental limitations, including (1) effectiveness due to the insufficient use of topology and consistency information and (2) scalability due to the non-convex formulation and repeated computationally costly loss calculation. In this paper, we propose a position-aware regularized optimal transport framework for network alignment named PARROT. To tackle the effectiveness issue, the proposed PARROT captures topology information by random walk with restart, with three carefully designed consistency regularization terms. To tackle the scalability issue, the regularized OT problem is decomposed into a series of convex subproblems and can be efficiently solved by the proposed constrained proximal point method with guaranteed convergence. Extensive experiments show that our algorithm achieves significant improvements in both effectiveness and scalability, outperforming the state-of-the-art network alignment methods and speeding up existing OT-based methods by up to 100 times.
FedEdge: Accelerating Edge-Assisted Federated Learning

Federated learning (FL) has been widely acknowledged as a promising solution to training machine learning (ML) model training with privacy preservation. To reduce the traffic overheads incurred by FL systems, edge servers have been included between clients and the parameter server to aggregate clients’ local models. Recent studies on this edge-assisted hierarchical FL scheme have focused on ensuring or accelerating model convergence by coping with various factors, e.g., uncertain network conditions, unreliable clients, heterogeneous compute resources, etc. This paper presents our three new discoveries of the edge-assisted hierarchical FL scheme: 1) it wastes significant time during its two-phase training rounds; 2) it does not recognize or utilize model diversity when producing a global model; and 3) it is vulnerable to model poisoning attacks. To overcome these drawbacks, we propose FedEdge, a novel edge-assisted hierarchical FL scheme that accelerates model training with asynchronous local federated training and adaptive model aggregation. Extensive experiments are conducted on two widely-used public datasets. The results demonstrate that, compared with state-of-the-art FL schemes, FedEdge accelerates model convergence by 1.14 × −3.20 ×, and improves model accuracy by 2.14% - 6.63%.
PaGE-Link: Path-based Graph Neural Network Explanation for Heterogeneous Link Prediction

Transparency and accountability have become major concerns for black-box machine learning (ML) models. Proper explanations for the model behavior increase model transparency and help researchers develop more accountable models. Graph neural networks (GNN) have recently shown superior performance in many graph ML problems than traditional methods, and explaining them has attracted increased interest. However, GNN explanation for link prediction (LP) is lacking in the literature. LP is an essential GNN task and corresponds to web applications like recommendation and sponsored search on web. Given existing GNN explanation methods only address node/graph-level tasks, we propose Path-based GNN Explanation for heterogeneous Link prediction (PaGE-Link) that generates explanations with connection interpretability, enjoys model scalability, and handles graph heterogeneity. Qualitatively, PaGE-Link can generate explanations as paths connecting a node pair, which naturally captures connections between the two nodes and easily transfer to human-interpretable explanations. Quantitatively, explanations generated by PaGE-Link improve AUC for recommendation on citation and user-item graphs by 9 - 35% and are chosen as better by 78.79% of responses in human evaluation.
Hierarchical Self-Attention Embedding for Temporal Knowledge Graph Completion

Temporal Knowledge Graph (TKG) is composed of a series of facts related to timestamps in the real world and has become the basis of many artificial intelligence applications. However, the existing TKG is usually incomplete. It has become a hot research task to infer missing facts based on existing facts in a TKG; namely, Temporal Knowledge Graph Completion (TKGC). The current mainstream TKGC models are embedded models that predict missing facts by representing entities, relations and timestamps as low-dimensional vectors. In order to deal with the TKG structure information, there are some models that try to introduce attention mechanism into the embedding process. But they only consider the structure information of entities or relations, and ignore the structure information of the whole TKG. Moreover, most of them usually treat timestamps as a general feature and cannot take advantage of the potential time series information of the timestamp. To solve these problems, wo propose a new Hierarchical Self-Attention Embedding (HSAE) model which inspired by self-attention mechanism and diachronic embedding technique. For structure information of the whole TKG, we divide the TKG into two layers: entity layer and relation layer, and then apply the self-attention mechanism to the entity layer and relation layer respectively to capture the structure information. For time series information of the timestamp, we capture them by combining positional encoding and diachronic embedding technique into the above two self-attention layers. Finally, we can get the embedded representation vectors of entities, relations and timestamps, which can be combined with other models for better results. We evaluate our model on three TKG datasets: ICEWS14, ICEWS05-15 and GDELT. Experimental results on the TKGC (interpolation) task demonstrate that our model achieves state-of-the-art results.
To Store or Not? Online Data Selection for Federated Learning with Limited Storage

Machine learning models have been deployed in mobile networks to deal with massive data from different layers to enable automated network management and intelligence on devices. To overcome high communication cost and severe privacy concerns of centralized machine learning, federated learning (FL) has been proposed to achieve distributed machine learning among networked devices. While the computation and communication limitation has been widely studied, the impact of on-device storage on the performance of FL is still not explored. Without an effective data selection policy to filter the massive streaming data on devices, classical FL can suffer from much longer model training time (4 ×) and significant inference accuracy reduction (7%), observed in our experiments. In this work, we take the first step to consider the online data selection for FL with limited on-device storage. We first define a new data valuation metric for data evaluation and selection in FL with theoretical guarantees for speeding up model convergence and enhancing final model accuracy, simultaneously. We further design ODE, a framework of Online Data sElection for FL, to coordinate networked devices to store valuable data samples. Experimental results on one industrial dataset and three public datasets show the remarkable advantages of ODE over the state-of-the-art approaches. Particularly, on the industrial dataset, ODE achieves as high as 2.5 × speedup of training time and 6% increase in inference accuracy, and is robust to various factors in practical environments.
Follow Us and Become Famous! Insights and Guidelines From Instagram Engagement Mechanisms

With 1.3 billion users, Instagram (IG) has become an essential business tool. IG influencer marketing, expected to generate $33.25 billion in 2022, encourages companies and influencers to create trending content. Various methods have been proposed for predicting a post's popularity, i.e., how much engagement (e.g., Likes) it will generate. However, these methods are limited: first, they focus on forecasting the likes, ignoring the number of comments, which became crucial in 2021. Secondly, studies often use biased or limited data. Third, researchers focused on Deep Learning models to increase predictive performance, which are difficult to interpret. As a result, end-users can only estimate engagement after a post is created, which is inefficient and expensive. A better approach is to generate a post based on what people and IG like, e.g., by following guidelines.
Set in Stone: Analysis of an Immutable Web3 Social Media Platform

There has been growing interest in the so-called “Web3” movement. This loosely refers to a mix of decentralized technologies, often underpinned by blockchain technologies. Among these, Web3 social media platforms have begun to emerge. These store all social interaction data (e.g., posts) on a public ledger, removing the need for centralized data ownership and management. But this comes at a cost, which some argue is prohibitively expensive. As an exemplar within this growing ecosytem, we explore memo.cash, a microblogging service built on the Bitcoin Cash (BCH) blockchain. We gather data for 24K users, 317K posts, 2.57M user actions, which have facilitated $6.75M worth of transactions. A particularly unique feature is that users must pay BCH tokens for each interaction (e.g., posting, following). We study how this may impact the social makeup of the platform. We therefore study memo.cash as both a social network and a transaction platform.
Rethinking Structural Encodings: Adaptive Graph Transformer for Node Classification Task

Graph Transformers have proved their advantages in graph data mining with elaborate Positional Encodings, especially in graph-level tasks. However, their application in the node classification task has not been fully exploited yet. In the node classification task, existing Graph Transformers with Positional Encodings are limited by the following issues: (i) PEs describing the node’s positional identities are insufficient for the node classification task on complex graphs, where a full portrayal of the local node property is needed. (ii) PEs for graphs are integrated with Transformers in a constant schema, resulting in the ignorance of local patterns that may vary among different nodes. In this paper, we propose Adaptive Graph Transformer (AGT) to tackle above issues. AGT consists of a Learnable Centrality Encoding and a Kernelized Local Structure Encoding. The two modules extract structural patterns from centrality and subgraph views in a learnable and scalable manner. Further, we design the Adaptive Transformer Block to adaptively integrate the attention scores and Structural Encodings in a node-specific manner. AGT achieves state-of-the-art performances on nine real-world web graphs (up to 1.6 million nodes). Furthermore, AGT shows outstanding results on two series of synthetic graphs with ranges of heterophily and noise ratios.
All Your Shops Are Belong to Us: Security Weaknesses in E-commerce Platforms

Software as a Service (SaaS) e-commerce platforms for merchants allow individual business owners to set up their online stores almost instantly. Prior work has shown that the checkout flows and payment integration of some e-commerce applications are vulnerable to logic bugs with serious financial consequences, e.g., allowing “shopping for free”. Apart from checkout and payment integration, vulnerabilities in other e-commerce operations have remained largely unexplored, even though they can have far more serious consequences, e.g., enabling “store takeover”. In this work, we design and implement a security evaluation framework to uncover security vulnerabilities in e-commerce operations beyond checkout/payment integration. We use this framework to analyze 32 representative e-commerce platforms, including web services of 24 commercial SaaS platforms and 15 associated Android apps, and 8 open source platforms; these platforms host over 10 million stores as approximated through Google dorks. We uncover several new vulnerabilities with serious consequences, e.g., allowing an attacker to take over all stores under a platform, and listing illegal products at a victim’s store—in addition to “shopping for free” bugs, without exploiting the checkout/payment process. We found 12 platforms vulnerable to store takeover (affecting 41000+ stores) and 6 platforms vulnerable to shopping for free (affecting 19000+ stores, approximated via Google dorks on Oct. 8, 2022). We have responsibly disclosed the vulnerabilities to all affected parties, and requested four CVEs (three assigned, and one is pending review).
Auto-HeG: Automated Graph Neural Network on Heterophilic Graphs

Graph neural architecture search (NAS) has gained popularity in automatically designing powerful graph neural networks (GNNs) with relieving human efforts. However, existing graph NAS methods mainly work under the homophily assumption and overlook another important graph property, i.e., heterophily, which exists widely in various real-world applications. To date, automated heterophilic graph learning with NAS is still a research blank to be filled in. Due to the complexity and variety of heterophilic graphs, the critical challenge of heterophilic graph NAS mainly lies in developing the heterophily-specific search space and strategy. Therefore, in this paper, we propose a novel automated graph neural network on heterophilic graphs, namely Auto-HeG, to automatically build heterophilic GNN models with expressive learning abilities. Specifically, Auto-HeG incorporates heterophily into all stages of automatic heterophilic graph learning, including search space design, supernet training, and architecture selection. Through the diverse message-passing scheme with joint micro-level and macro-level designs, we first build a comprehensive heterophilic GNN search space, enabling Auto-HeG to integrate complex and various heterophily of graphs. With a progressive supernet training strategy, we dynamically shrink the initial search space according to layer-wise variation of heterophily, resulting in a compact and efficient supernet. Taking a heterophily-aware distance criterion as the guidance, we conduct heterophilic architecture selection in the leave-one-out pattern, so that specialized and expressive heterophilic GNN architectures can be derived. Extensive experiments illustrate the superiority of Auto-HeG in developing excellent heterophilic GNNs to human-designed models and graph NAS models.
HyConvE: A Novel Embedding Model for Knowledge Hypergraph Link Prediction with Convolutional Neural Networks

Knowledge hypergraph embedding, which projects entities and n-ary relations into a low-dimensional continuous vector space to predict missing links, remains a challenging area to be explored despite the ubiquity of n-ary relational facts in the real world. Currently, knowledge hypergraph link prediction methods are essentially simple extensions of those used in knowledge graphs, where n-ary relational facts are decomposed into different subelements. Convolutional neural networks have been shown to have remarkable information extraction capabilities in previous work on knowledge graph link prediction. In this paper, we propose a novel embedding-based knowledge hypergraph link prediction model named HyConvE, which exploits the powerful learning ability of convolutional neural networks for effective link prediction. Specifically, we employ 3D convolution to capture the deep interactions of entities and relations to efficiently extract explicit and implicit knowledge in each n-ary relational fact without compromising its translation property. In addition, appropriate relation and position-aware filters are utilized sequentially to perform two-dimensional convolution operations to capture the intrinsic patterns and position information in each n-ary relation, respectively. Extensive experimental results on real datasets of knowledge hypergraphs and knowledge graphs demonstrate the superior performance of HyConvE compared with state-of-the-art baselines.
Recommendation with Causality enhanced Natural Language Explanations

Explainable recommendation has recently attracted increasing attention from both academic and industry communities. Among different explainable strategies, generating natural language explanations is an important method, which can deliver more informative, flexible and readable explanations to facilitate better user decisions. Despite the effectiveness, existing models are mostly optimized based on the observed datasets, which can be skewed due to the selection or exposure bias. To alleviate this problem, in this paper, we formulate the task of explainable recommendation with a causal graph, and design a causality enhanced framework to generate unbiased explanations. More specifically, we firstly define an ideal unbiased learning objective, and then derive a tractable loss for the observational data based on the inverse propensity score (IPS), where the key is a sample re-weighting strategy for equalizing the loss and ideal objective in expectation. Considering that the IPS estimated from the sparse and noisy recommendation datasets can be inaccurate, we introduce a fault tolerant mechanism by minimizing the maximum loss induced by the sample weights near the IPS. For more comprehensive modeling, we further analyze and infer the potential latent confounders induced by the complex and diverse user personalities. We conduct extensive experiments by comparing with the state-of-the-art methods based on three real-world datasets to demonstrate the effectiveness of our method.
Improving Content Retrievability in Search with Controllable Query Generation

An important goal of online platforms is to enable content discovery, i.e. allow users to find a catalog entity they were not familiar with. A pre-requisite to discover an entity, e.g. a book, with a search engine is that the entity is retrievable, i.e. there are queries for which the system will surface such entity in the top results. However, machine-learned search engines have a high retrievability bias, where the majority of the queries return the same entities. This happens partly due to the predominance of narrow intent queries, where users create queries using the title of an already known entity, e.g. in book search “harry potter”. The amount of broad queries where users want to discover new entities, e.g. in music search “chill lyrical electronica with an atmospheric feeling to it”, and have a higher tolerance to what they might find, is small in comparison. We focus here on two factors that have a negative impact on the retrievability of the entities (I) the training data used for dense retrieval models and (II) the distribution of narrow and broad intent queries issued in the system. We propose CtrlQGen, a method that generates queries for a chosen underlying intent—narrow or broad. We can use CtrlQGen to improve factor (I) by generating training data for dense retrieval models comprised of diverse synthetic queries. CtrlQGen can also be used to deal with factor (II) by suggesting queries with broader intents to users. Our results on datasets from the domains of music, podcasts, and books reveal that we can significantly decrease the retrievability bias of a dense retrieval model when using CtrlQGen. First, by using the generated queries as training data for dense models we make 9% of the entities retrievable—go from zero to non-zero retrievability. Second, by suggesting broader queries to users, we can make 12% of the entities retrievable in the best case.
TMMDA: A New Token Mixup Multimodal Data Augmentation for Multimodal Sentiment Analysis

Existing methods for Multimodal Sentiment Analysis (MSA) mainly focus on integrating multimodal data effectively on limited multimodal data. Learning more informative multimodal representation often relies on large-scale labeled datasets, which are difficult and unrealistic to obtain. To learn informative multimodal representation on limited labeled datasets as more as possible, we proposed TMMDA for MSA, a new Token Mixup Multimodal Data Augmentation, which first generates new virtual modalities from the mixed token-level representation of raw modalities, and then enhances the representation of raw modalities by utilizing the representation of the generated virtual modalities. To preserve semantics during virtual modality generation, we propose a novel cross-modal token mixup strategy based on the generative adversarial network. Extensive experiments on two benchmark datasets, i.e., CMU-MOSI and CMU-MOSEI, verify the superiority of our model compared with several state-of-the-art baselines. The code is available at https://github.com/xiaobaicaihhh/TMMDA.