Reviewerly/exHarmony huggingface dataset:
Similarity Preserving Adversarial Graph Contrastive Learning

Recent works demonstrate that GNN models are vulnerable to adversarial attacks, which refer to imperceptible perturbation on the graph structure and node features. Among various GNN models, graph contrastive learning (GCL) based methods specifically suffer from adversarial attacks due to their inherent design that highly depends on the self-supervision signals derived from the original graph, which however already contains noise when the graph is attacked. To achieve adversarial robustness against such attacks, existing methods adopt adversarial training (AT) to the GCL framework, which considers the attacked graph as an augmentation under the GCL framework. However, we find that existing adversarially trained GCL methods achieve robustness at the expense of not being able to preserve the node feature similarity. In this paper, we propose a similarity-preserving adversarial graph contrastive learning (SP-AGCL) framework that contrasts the clean graph with two auxiliary views of different properties (i.e., the node similarity-preserving view and the adversarial view). Extensive experiments demonstrate that SP-AGCL achieves a competitive performance on several downstream tasks, and shows its effectiveness in various scenarios, e.g., a network with adversarial attacks,noisy labels, and heterophilous neighbors. Our code is available at https://github.com/yeonjun-in/torch-SP-AGCL.
M5: Multi-Modal Multi-Interest Multi-Scenario Matching for Over-the-Top Recommendation

Matching preferred shows to the subscribers is extremely important in the Over-the-Top (OTT) platforms. The existing methods did not adequately consider the characteristics of the OTT services, i.e., rich meta information, diverse user interests, and mixed recommendation scenarios, leading to sub-optimal performance. This paper introduces the Multi-Modal Multi-Interest Multi-Scenario Matching (M5) for the OTT recommendation to fully exploit these attributes. A multi-modal embedding layer is first introduced to transform the show IDs into both ID embeddings initialized randomly and content graph (CG) embeddings derived from the node representations pre-trained on a metagraph. To segregate the semantics between ID and CG embeddings, M5 exploits the mirrored two-tower modeling in the subsequent layers for efficiency and effectiveness. Specifically, a multi-interest extraction layer is proposed separately on ID and CG behaviors to model users' coarse-grained and fine-grained interests through behavioral categorization, subsidiary decoration, masked-language-modeling augmented self-attention modeling and subsidiary-intensity interest calibration. Facing the inherent diverse scenarios, M5 distinguishes the scenario differences at both feature and model levels, which crosses features with the scenario indicators and employs Split Mixture-of-Experts to generate the ID, and CG user embeddings. Finally, a weighted candidate matching layer is established to calculate the ID- and CG-oriented user-item preferences and then merge into a hybrid score with dynamic weighting. The extensive online and offline experiments over two real-world OTT platforms Hulu and Disney+ reveal that M5 significantly outperforms the previous state-of-the-art and online matching algorithms over various scenarios, indicating the effectiveness and robustness of the proposed method. M5 has been fully deployed on the main traffic of the most popular "For You'' sets of both platforms, continuously enhancing the user experience for hundreds of millions of subscribers every day and steadily increasing business revenue.
Robust Positive-Unlabeled Learning via Noise Negative Sample Self-correction

Learning from positive and unlabeled data is known as positive-unlabeled (PU) learning in literature and has attracted much attention in recent years. One common approach in PU learning is to sample a set of pseudo-negatives from the unlabeled data using ad-hoc thresholds so that conventional supervised methods can be applied with both positive and negative samples. Owing to the label uncertainty among the unlabeled data, errors of misclassifying unlabeled positive samples as negative samples inevitably appear and may even accumulate during the training processes. Those errors often lead to performance degradation and model instability. To mitigate the impact of label uncertainty and improve the robustness of learning with positive and unlabeled data, we propose a new robust PU learning method with a training strategy motivated by the nature of human learning: easy cases should be learned first. Similar intuition has been utilized in curriculum learning to only use easier cases in the early stage of training before introducing more complex cases. Specifically, we utilize a novel ''hardness'' measure to distinguish unlabeled samples with a high chance of being negative from unlabeled samples with large label noise. An iterative training strategy is then implemented to fine-tune the selection of negative samples during the training process in an iterative manner to include more ''easy'' samples in the early stage of training. Extensive experimental validations over a wide range of learning tasks show that this approach can effectively improve the accuracy and stability of learning with positive and unlabeled data. Our code is available at https://github.com/woriazzc/Robust-PU.
FLAMES2Graph: An Interpretable Federated Multivariate Time Series Classification Framework

Increasing privacy concerns have led to decentralized and federated machine learning techniques that allow individual clients to consult and train models collaboratively without sharing private information. Some of these applications, such as medical and healthcare, require the final decisions to be interpretable. One common form of data in these applications is multivariate time series, where deep neural networks, especially convolutional neural networks based approaches, have established excellent performance in their classification tasks. However, promising results and performance of deep learning models are a black box, and their decisions cannot always be guaranteed and trusted. While several approaches address the interpretability of deep learning models for multivariate time series data in a centralized environment, less effort has been made in a federated setting. In this work, we introduce FLAMES2Graph, a new horizontal federated learning framework designed to interpret the deep learning decisions of each client. FLAMES2Graph extracts and visualizes those input subsequences that are highly activated by a convolutional neural network. Besides, an evolution graph is created to capture the temporal dependencies between the extracted distinct subsequences. The federated learning clients only share this temporal evolution graph with the centralized server instead of trained model weights to create a global evolution graph. Our extensive experiments on various datasets from well-known multivariate benchmarks indicate that the FLAMES2Graph framework significantly outperforms other state-of-the-art federated methods while keeping privacy and augmenting network decision interpretation.
MIDLG: Mutual Information based Dual Level GNN for Transaction Fraud Complaint Verification

"Transaction fraud" complaint verification, i.e., verifying whether a transaction corresponding to a complaint is fraudulent, is particularly critical to prevent economic loss. Compared with traditional fraud pre-transaction detection, complaint verification puts forward higher requirements: 1)an individual tends to exhibit different identities in different complaints, e.g., complainant or respondent, requiring the model to capture identity-related representations corresponding to the complaint; 2)the fraud ways evolve frequently to confront detection, requiring the model to perform stably under different fraud ways. Previous methods mainly focused on fraud pre-transaction detection, utilizing the historical information of users or conduct message passing based GNNs on relationship networks. However, they rarely consider capturing various identity-related representations and ignore the evolution of fraud ways, leading to failure in complaint verification. To address the above challenges, we propose the mutual information based dual level graph neural network, namely MIDLG, which defines a complaint as a super-node consisting of involved individuals, and characterizes the individual over node-level and super-node-level. Furthermore, the mutual information minimization objective is proposed based on "complaint verification-causal graph" to decouple the model prediction from relying on specific fraud ways, and thus achieve stability. MIDLG achieves SOTA results through extensive experiments in complaint verification on WeChat Finance, one online payment service serving more than 600 million users in China.
Multi-channel Integrated Recommendation with Exposure Constraints

Integrated recommendation, which aims at jointly recommending heterogeneous items from different channels in a main feed, has been widely applied to various online platforms. Though attractive, integrated recommendation requires the ranking methods to migrate from conventional user-item models to the new user-channel-item paradigm in order to better capture users' preferences on both item and channel levels. Moreover, practical feed recommendation systems usually impose exposure constraints on different channels to ensure user experience. This leads to greater difficulty in the joint ranking of heterogeneous items. In this paper, we investigate the integrated recommendation task with exposure constraints in practical recommender systems. Our contribution is forth-fold. First, we formulate this task as a binary online linear programming problem and propose a two-layer framework named Multi-channel Integrated Recommendation with Exposure Constraints~(MIREC) to obtain the optimal solution. Second, we propose an efficient online allocation algorithm to determine the optimal exposure assignment of different channels from a global view of all user requests over the entire time horizon. We prove that this algorithm reaches the optimal point under a regret bound of O (√T) with linear complexity. Third, we propose a series of collaborative models to determine the optimal layout of heterogeneous items at each user request. The joint modeling of user interests, cross-channel correlation, and page context in our models aligns more with the browsing nature of feed products than existing models. Finally, we conduct extensive experiments on both offline datasets and online A/B tests to verify the effectiveness of MIREC. The proposed framework has now been implemented on the homepage of Taobao to serve the main traffic.
Graph Neural Networks: Foundation, Frontiers and Applications

The field of graph neural networks (GNNs) has seen rapid and incredible strides over the recent years. Graph neural networks, also known as deep learning on graphs, graph representation learning, or geometric deep learning, have become one of the fastest-growing research topics in machine learning, especially deep learning. However, as the field rapidly grows, it has been extremely challenging to gain a global perspective of the developments of GNNs. Therefore, we feel the urgency to bridge the above gap and have a comprehensive tutorial on this fast-growing yet challenging topic.
MOIT: A Novel task for mining opinions towards implicit targets

The extraction of opinions and their corresponding targets has gained significant interest recently, as it offers valuable insights into Opinion Mining (OM) at a granular level. Opinion and target terms to be extracted by existing OM tasks need to be explicitly present in reviews. Targets that are not present but implied in contextual semantics, are neglected by existing OM tasks, even though an investigation reported that about 60% of reviews contain implicit targets. To enable implicit target extraction, a novel task named Mining Opinions towards Implicit Targets (MOIT) under the fine-grained OM, is proposed to extract both opinions and their corresponding implicit targets, enabling a more comprehensive analysis of reviews. To set up the basis for follow-up research on MOIT, two large-scale datasets were constructed as resources in two languages, where the Chinese dataset was built from scratch via a standard human annotation process, and the English dataset was built semi-automatically through machine translation and manual checking. Furthermore, three baseline models adapting three representative paradigms of information extraction, namely sequence labeling, question answering, and text generation, were proposed to solve MOIT. Extensive experiments demonstrated the effectiveness of the models. The proposed MOIT task extends the field of OM research, and the datasets and models establish a foundation for future studies in this area.
Efficient, secure and verifiable outsourcing scheme for SVD-based collaborative filtering recommender system

With the explosive growth of e-commerce, recommender systems are widely used to generate personalized recommendation for customers. SVD-based collaborative filtering and its variants are highly accurate and scalable approaches to recommender systems. Due to the heavy computation of SVD-based collaborative filtering, outsourcing the computation is an efficient solution to reduce computational complexity. In this paper, we propose an efficient, secure and verifiable outsourcing scheme for SVD-based collaborative filtering recommender system. We use symmetric block diagonal matrices as seeds to generate secret keys, which are novel orthogonal sparse matrices to blind the target matrices of SVD. Security analysis shows that our scheme can protect the privacy of both the input and output and efficiency analysis shows that our scheme is (3m2+3n2+5mn)/(m3+n3) efficient compared to fully local algorithm. In our scheme, we also create a verification approach that is capable of detecting misbehavior from a cloud server with probability (1−12n). The experiment shows that the client achieves significant computational savings and the recommendation accuracy of the scheme is nearly as good as that of the fully local algorithm.
Optimized compressed sensing for communication efficient federated learning

In recent years, data privacy preservation has received increased attention in artificial intelligence. Federated learning, as a paradigm for privacy-preserving machine learning, can considerably reduce the risk of privacy leakage by training models on local data. In federated learning, however, the clients and server must interact twice in each round of federated training, consuming abundant communication resources. To address the above issue, this work proposes an enhanced compressed sensing federated learning algorithm, which compresses and reconstructs the local network models trained on the clients using compressed sensing. To enhance the accuracy of the reconstructed models, we optimize the measurement matrix in compressed sensing using a genetic algorithm. In addition, we suggest an interleaving training and reconstruction method to improve the learning performance of the compressed models. Through a large number of experiments, we demonstrate that the proposed method is capable of maintaining high learning accuracy while accomplishing a large compression ratio for deep network models in federated learning.
Auto weighted robust dual graph nonnegative matrix factorization for multiview clustering

Recent multi-data fusion techniques have inspired multi-view clustering study, which has become a popular topic in machine learning. One successful way of solving multi-view clustering is nonnegative matrix factorization. This paper proposes an auto weighted robust dual graph nonnegative matrix factorization (ARDNMF), where ℓ2,1 norm is adopted to make ARDNMF robust as data contain noise and outliers. Through dual graphs, structure information in data space and feature space is considered. Moreover, an auto weighted assignment method is introduced in ARDNMF to assign appropriate weights to views. Effective updating rules are given. Experiments on real datasets show ARDNMF superiors to state-of-the-art algorithms in terms of clustering results.
An AUC-maximizing classifier for skewed and partially labeled data with an application in clinical prediction modeling

Partially labeled and skewed datasets are common in many applications including healthcare, due to the high costs and time constraints of data collection and annotation. However, training machine learning classifiers on such data can undermine their prediction performances. In this paper, we propose a novel classifier to address this problem by focusing on the Area Under the Curve (AUC), which is widely recognized as a more robust performance metric for skewed datasets than other metrics such as accuracy and error rate. We introduce a new classifier called PSVM-AUC Maximizer (PSVM-AUCMax) which is based on Proximal Support Vector Machines (PSVM) and directly maximizes a new AUC-based metric in its learning objective. PSVM-AUCMax has several merits. First, by directly integrating the maximization of the proposed AUC-based metric, PSVM-AUCMax can be proved to have the enhanced generalization capability on the partially labeled and skewed dataset. Second, it simplifies the model selection process with fewer tuning hyperparameters. Third, PSVM-AUCMax’s analytical solution remains the same form as traditional PSVM, preserving its advantages such as fast incremental updating in incremental learning scenarios. The efficacy of PSVM-AUCMax has been demonstrated through extensive experiments on several public datasets and a healthcare case study using data collected at the US Mayo Clinic. In the healthcare case study, we utilized PSVM-AUCMax to develop a clinical prediction model for forecasting composite outcomes in hospitalized COVID-19 patients which yielded promising results.
A novel shape augmentation approach in training neural networks using Branch Length Similarity entropy

This study proposes a new approach for training deep neural networks for shape classification. The approach uses Branch Length Similarity (BLS) entropy, a measure defined for simple networks. The BLS entropy profile is obtained by computing entropy values in either a clockwise or counterclockwise direction of the boundary pixels forming the shape. Instead of directly training the deep neural network on the shape, the BLS entropy profile is trained as a representation of the shape. To assess the effectiveness of this approach, we conducted experiments using the MPEG-7 dataset. We selected 40 shape classes, each consisting of 10 similar shapes, and created 20 deformed versions per shape by modifying their aspect ratios. We trained GoogLeNet on these deformed shapes (referred to as GoogLeNet_Shapes), as well as on an image of the entropy profile derived from the deformed shapes (referred to as GoogLeNet_Entropy) and a circularly shifted image of the profile (referred to as GoogLeNet_Shifted). During the classification task on rotated shapes that were not included in the training set, GoogLeNet_Shapes achieved a classification accuracy of 41.66%, GoogLeNet_Entropy achieved 45.47% accuracy, and GoogLeNet_Shifted achieved 82.26% accuracy. Similarly, we compared the test accuracy of GoogLeNet_Shape, GoogLeNet_Entropy, and GoogLeNet_Shifted for rotation and scaling transformations. Interestingly, we observed that GoogLeNet_Shape performed well for the transformations it was trained on, while GoogLeNet_Shifted exhibited superior performance for other transformations that were not part of the training data, all under the condition of the same amount of data.
Certification and applications of quantum nonlocal correlations

Abstract Entanglement and Einstein–Podolsky–Rosen (EPR) steering are nonlocal quantum correlations, which are relevant resources for quantum information protocols. EPR steering, or quantum steering, refers to the correlation where a party might ‘steer’, or modify, the state of another, which is spatially separated. Entanglement is a symmetric resource while steering is asymmetrical, since it depends on the direction of the effect. Due to these different characteristics and the therefore different possible applications, there has been both theoretical and experimental research on forms to certify the distinct quantum nonlocal correlations. In recent years, alongside the investigation on quantum correlations between two systems, there has been a great interest in investigating multipartite/multimode entanglement as well as steering, since they include a high dimension and it may be possible to store more information than in a single qubit. In this review, we will summarize the different criteria and measures that have been developed for the characterization of these two kinds of correlations. We first focus on bipartite entanglement and steering. We then review the progress that has been made in the investigation of multipartite quantum correlations. We revise the theoretical work in quantum nonlocal correlation witnesses and measures, which respectively allow one to certify that the system is entangled or presents EPR steering, and give a quantification of the content of these correlations in the system. Then, we briefly review the experiments that have been designed and that demonstrate multipartite quantum correlations. We also include applications in quantum information protocols, in particular in quantum teleportation and quantum cryptography.
A Semi-supervised Approach for a Better Translation of Sentiment in Dialectical Arabic UGT

In the online world, Machine Translation (MT) systems are extensively used to translate User-Generated Text (UGT) such as reviews, tweets, and social media posts, where the main message is often the author's positive or negative attitude towards the topic of the text.However, MT systems still lack accuracy in some low-resource languages and sometimes make critical translation errors that completely flip the sentiment polarity of the target word or phrase and hence delivers a wrong affect message.This is particularly noticeable in texts that do not follow common lexico-grammatical standards such as the dialectical Arabic (DA) used on online platforms.In this research, we aim to improve the translation of sentiment in UGT written in the dialectical versions of the Arabic language to English.Given the scarcity of gold-standard parallel data for DA-EN in the UGT domain, we introduce a semi-supervised approach that exploits both monolingual and parallel data for training an NMT system initialised by a cross-lingual language model trained with supervised and unsupervised modeling objectives.We assess the accuracy of sentiment translation by our proposed system through a numerical 'sentimentcloseness' measure as well as human evaluation.We will show that our semi-supervised MT system can significantly help with correcting sentiment errors detected in the online translation of dialectical Arabic UGT.
Cross-lingual Text-to-SQL Semantic Parsing with Representation Mixup

We focus on the cross-lingual Text-to-SQL semantic parsing task, where the parsers are expected to generate SQL for non-English utterances based on English database schemas.Intuitively, English translation as side information is an effective way to bridge the language gap, but noise introduced by the translation system may affect parser effectiveness.In this work, we propose a Representation Mixup Framework (REX) for effectively exploiting translations in the cross-lingual Text-to-SQL task.Particularly, it uses a general encoding layer, a transition layer, and a target-centric layer to properly guide the information flow of the English translation.Experimental results on CSPIDER and VSPIDER show that our framework can benefit from cross-lingual training and improve the effectiveness of semantic parsers, achieving state-of-the-art performance.3 https://taolusi.github.io/CSpider-explorer