pwc-archive/papers-with-abstracts huggingface dataset:
A fully Bayesian sparse polynomial chaos expansion approach with joint priors on the coefficients and global selection of terms
Authors: Paul-Christian Bürkner, Ilja Kröker, Sergey Oladyshkin, Wolfgang Nowak
Date: 2022-04-12 00:00:00
Tasks: Uncertainty Quantification, Variable Selection

Polynomial chaos expansion (PCE) is a versatile tool widely used in uncertainty quantification and machine learning, but its successful application depends strongly on the accuracy and reliability of the resulting PCE-based response surface. High accuracy typically requires high polynomial degrees, demanding many training points especially in high-dimensional problems through the curse of dimensionality. So-called sparse PCE concepts work with a much smaller selection of basis polynomials compared to conventional PCE approaches and can overcome the curse of dimensionality very efficiently, but have to pay specific attention to their strategies of choosing training points. Furthermore, the approximation error resembles an uncertainty that most existing PCE-based methods do not estimate. In this study, we develop and evaluate a fully Bayesian approach to establish the PCE representation via joint shrinkage priors and Markov chain Monte Carlo. The suggested Bayesian PCE model directly aims to solve the two challenges named above: achieving a sparse PCE representation and estimating uncertainty of the PCE itself. The embedded Bayesian regularizing via the joint shrinkage prior allows using higher polynomial degrees for given training points due to its ability to handle underdetermined situations, where the number of considered PCE coefficients could be much larger than the number of available training points. We also explore multiple variable selection methods to construct sparse PCE expansions based on the established Bayesian representations, while globally selecting the most meaningful orthonormal polynomials given the available training data. We demonstrate the advantages of our Bayesian PCE and the corresponding sparsity-inducing methods on several benchmarks.
A pipeline and comparative study of 12 machine learning models for text classification
Authors: Annalisa Occhipinti, Louis Rogers, Claudio Angione
Date: 2022-04-04 00:00:00
Tasks: BIG-bench Machine Learning, Classification, text-classification, Text Classification

Text-based communication is highly favoured as a communication method, especially in business environments. As a result, it is often abused by sending malicious messages, e.g., spam emails, to deceive users into relaying personal information, including online accounts credentials or banking details. For this reason, many machine learning methods for text classification have been proposed and incorporated into the services of most email providers. However, optimising text classification algorithms and finding the right tradeoff on their aggressiveness is still a major research problem. We present an updated survey of 12 machine learning text classifiers applied to a public spam corpus. A new pipeline is proposed to optimise hyperparameter selection and improve the models' performance by applying specific methods (based on natural language processing) in the preprocessing stage. Our study aims to provide a new methodology to investigate and optimise the effect of different feature sizes and hyperparameters in machine learning classifiers that are widely used in text classification problems. The classifiers are tested and evaluated on different metrics including F-score (accuracy), precision, recall, and run time. By analysing all these aspects, we show how the proposed pipeline can be used to achieve a good accuracy towards spam filtering on the Enron dataset, a widely used public email corpus. Statistical tests and explainability techniques are applied to provide a robust analysis of the proposed pipeline and interpret the classification outcomes of the 12 machine learning models, also identifying words that drive the classification results. Our analysis shows that it is possible to identify an effective machine learning model to classify the Enron dataset with an F-score of 94%.
CARCA: Context and Attribute-Aware Next-Item Recommendation via Cross-Attention
Authors: Ahmed Rashed, Shereen Elsayed, Lars Schmidt-Thieme
Date: 2022-04-04 00:00:00
Tasks: Attribute, Recommendation Systems, Sequential Recommendation

In sparse recommender settings, users' context and item attributes play a crucial role in deciding which items to recommend next. Despite that, recent works in sequential and time-aware recommendations usually either ignore both aspects or only consider one of them, limiting their predictive performance. In this paper, we address these limitations by proposing a context and attribute-aware recommender model (CARCA) that can capture the dynamic nature of the user profiles in terms of contextual features and item attributes via dedicated multi-head self-attention blocks that extract profile-level features and predicting item scores. Also, unlike many of the current state-of-the-art sequential item recommendation approaches that use a simple dot-product between the most recent item's latent features and the target items embeddings for scoring, CARCA uses cross-attention between all profile items and the target items to predict their final scores. This cross-attention allows CARCA to harness the correlation between old and recent items in the user profile and their influence on deciding which item to recommend next. Experiments on four real-world recommender system datasets show that the proposed model significantly outperforms all state-of-the-art models in the task of item recommendation and achieving improvements of up to 53% in Normalized Discounted Cumulative Gain (NDCG) and Hit-Ratio. Results also show that CARCA outperformed several state-of-the-art dedicated image-based recommender systems by merely utilizing image attributes extracted from a pre-trained ResNet50 in a black-box fashion.
Graph Enhanced BERT for Query Understanding
Authors: Juanhui Li, Yao Ma, Wei Zeng, Suqi Cheng, Jiliang Tang, Shuaiqiang Wang, Dawei Yin
Date: 2022-04-03 00:00:00

Query understanding plays a key role in exploring users' search intents and facilitating users to locate their most desired information. However, it is inherently challenging since it needs to capture semantic information from short and ambiguous queries and often requires massive task-specific labeled data. In recent years, pre-trained language models (PLMs) have advanced various natural language processing tasks because they can extract general semantic information from large-scale corpora. Therefore, there are unprecedented opportunities to adopt PLMs for query understanding. However, there is a gap between the goal of query understanding and existing pre-training strategies -- the goal of query understanding is to boost search performance while existing strategies rarely consider this goal. Thus, directly applying them to query understanding is sub-optimal. On the other hand, search logs contain user clicks between queries and urls that provide rich users' search behavioral information on queries beyond their content. Therefore, in this paper, we aim to fill this gap by exploring search logs. In particular, to incorporate search logs into pre-training, we first construct a query graph where nodes are queries and two queries are connected if they lead to clicks on the same urls. Then we propose a novel graph-enhanced pre-training framework, GE-BERT, which can leverage both query content and the query graph. In other words, GE-BERT can capture both the semantic information and the users' search behavioral information of queries. Extensive experiments on various query understanding tasks have demonstrated the effectiveness of the proposed framework.
Bayesian Negative Sampling for Recommendation
Authors: Bin Liu, Bang Wang
Date: 2022-04-02 00:00:00
Tasks: Collaborative Filtering, Contrastive Learning, Informativeness

How to sample high quality negative instances from unlabeled data, i.e., negative sampling, is important for training implicit collaborative filtering and contrastive learning models. Although previous studies have proposed some approaches to sample informative instances, few has been done to discriminating false negative from true negative for unbiased negative sampling. On the basis of our order relation analysis of negatives' scores, we first derive the class conditional density of true negatives and that of false negatives. We next design a Bayesian classifier for negative classification, from which we define a model-agnostic posterior probability estimate of an instance being true negative as a quantitative negative signal measure. We also propose a Bayesian optimal sampling rule to sample high-quality negatives. The proposed Bayesian Negative Sampling (BNS) algorithm has a linear time complexity. Experimental studies validate the superiority of BNS over the peers in terms of better sampling quality and better recommendation performance.
Optimizing generalized Gini indices for fairness in rankings
Authors: Virginie Do, Nicolas Usunier
Date: 2022-04-02 00:00:00
Tasks: Fairness, Recommendation Systems

There is growing interest in designing recommender systems that aim at being fair towards item producers or their least satisfied users. Inspired by the domain of inequality measurement in economics, this paper explores the use of generalized Gini welfare functions (GGFs) as a means to specify the normative criterion that recommender systems should optimize for. GGFs weight individuals depending on their ranks in the population, giving more weight to worse-off individuals to promote equality. Depending on these weights, GGFs minimize the Gini index of item exposure to promote equality between items, or focus on the performance on specific quantiles of least satisfied users. GGFs for ranking are challenging to optimize because they are non-differentiable. We resolve this challenge by leveraging tools from non-smooth optimization and projection operators used in differentiable sorting. We present experiments using real datasets with up to 15k users and items, which show that our approach obtains better trade-offs than the baselines on a variety of recommendation tasks and fairness criteria.
Decentralized Collaborative Learning Framework for Next POI Recommendation
Authors: Jing Long, Tong Chen, Nguyen Quoc Viet Hung, Hongzhi Yin
Date: 2022-03-30 00:00:00
Tasks: Privacy Preserving

Next Point-of-Interest (POI) recommendation has become an indispensable functionality in Location-based Social Networks (LBSNs) due to its effectiveness in helping people decide the next POI to visit. However, accurate recommendation requires a vast amount of historical check-in data, thus threatening user privacy as the location-sensitive data needs to be handled by cloud servers. Although there have been several on-device frameworks for privacy-preserving POI recommendations, they are still resource-intensive when it comes to storage and computation, and show limited robustness to the high sparsity of user-POI interactions. On this basis, we propose a novel decentralized collaborative learning framework for POI recommendation (DCLR), which allows users to train their personalized models locally in a collaborative manner. DCLR significantly reduces the local models' dependence on the cloud for training, and can be used to expand arbitrary centralized recommendation models. To counteract the sparsity of on-device user data when learning each local model, we design two self-supervision signals to pretrain the POI representations on the server with geographical and categorical correlations of POIs. To facilitate collaborative learning, we innovatively propose to incorporate knowledge from either geographically or semantically similar users into each local model with attentive aggregation and mutual information maximization. The collaborative learning process makes use of communications between devices while requiring only minor engagement from the central server for identifying user groups, and is compatible with common privacy preservation mechanisms like differential privacy. We evaluate DCLR with two real-world datasets, where the results show that DCLR outperforms state-of-the-art on-device frameworks and yields competitive results compared with centralized counterparts.
Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation
Authors: Chao Chen, Haoyu Geng, Nianzu Yang, Junchi Yan, Daiyue Xue, Jianping Yu, Xiaokang Yang
Date: 2022-03-30 00:00:00
Tasks: Dynamic Link Prediction, Sequential Recommendation

User interests are usually dynamic in the real world, which poses both theoretical and practical challenges for learning accurate preferences from rich behavior data. Among existing user behavior modeling solutions, attention networks are widely adopted for its effectiveness and relative simplicity. Despite being extensively studied, existing attentions still suffer from two limitations: i) conventional attentions mainly take into account the spatial correlation between user behaviors, regardless the distance between those behaviors in the continuous time space; and ii) these attentions mostly provide a dense and undistinguished distribution over all past behaviors then attentively encode them into the output latent representations. This is however not suitable in practical scenarios where a user's future actions are relevant to a small subset of her/his historical behaviors. In this paper, we propose a novel attention network, named self-modulating attention, that models the complex and non-linearly evolving dynamic user preferences. We empirically demonstrate the effectiveness of our method on top-N sequential recommendation tasks, and the results on three large-scale real-world datasets show that our model can achieve state-of-the-art performance.
heterogeneous temporal graph transformer: an intelligent system for evolving android malware detection
Authors: Yujie Fan, Mingxuan Ju, Shifu Hou, Yanfang Ye, Wenqiang Wan, Kui Wang, Yinming Mei, Qi Xiong
Date: 2021-08-14 00:00:00
Proceeding: KDD 2021 8
Tasks: Android Malware Detection, Malware Detection, Mobile Security, Representation Learning

The explosive growth and increasing sophistication of Android malware call for new defensive techniques to protect mobile users against novel threats. To address this challenge, in this paper, we propose and develop an intelligent system named Dr.Droid to jointly model malware propagation and evolution for their detection at the first attempt. In Dr.Droid, we first exploit higher-level semantic and social relations within the ecosystem (e.g., app-market, app-developer, market-developer relations etc.) to characterize app propagation patterns; and then we present a structured heterogeneous graph to model the complex relations among different types of entities. To capture malware evolution, we further consider the temporal dependence and introduce a heterogeneous temporal graph to jointly model malware propagation and evolution by considering heterogeneous spatial dependencies with temporal dimensions. Afterwards, we propose a novel heterogeneous temporal graph transformer framework (denoted as HTGT) to integrate both spatial and temporal dependencies while preserving the heterogeneity to learn node representations for malware detection. Specifically, in our proposed HTGT, to preserve the heterogeneity, we devise a heterogeneous spatial transformer to derive heterogeneous attentions over each node and edge to learn dedicated representations for different types of entities and relations; to model temporal dependencies, we design a temporal transformer into the HTGT to attentively aggregate its historical sequences of a given node (e.g., app); the two transformers work in an iterative manner for representation learning. Promising experimental results based on the large-scale sample collections from anti-malware industry demonstrate the performance of Dr.Droid, by comparison with state-of-the-art baselines and popular mobile security products.
QR factorization-based sampling set selection for bandlimited graph signals
Authors: Yoon Hak Kim
Date: 2020-10-01 00:00:00
Proceeding: Signal Processing 2020 10

考虑采样图上的顶点或节点的问题，其中图上的信号是从采样节点上的信号值进行卷积和重构的。注意，对节点子集进行采样相当于构造采样矩阵，该矩阵由从变异算子（例如，图拉普拉斯算子）的特征向量矩阵中选择的相应行组成，我们首先将重构误差表示为采样矩阵伪逆的Frobenius矩阵范数，并通过QR分解来计算重构误差，从而提出一个简单的贪婪采样准则。我们证明了在高信噪比情况下，利用近似超模的概念，所提出的算法可以达到近似最优，并通过与针对传感器选择问题开发的近似最优方法进行比较，对所提出的方法进行了评估。我们还分析了该算法在运算计数方面的复杂性，并与现有的贪婪算法进行了比较，包括矩阵子集选择算法，因为图形信号的采样也是通过从特征向量矩阵的转置中选择列的子集来完成的。最后，我们通过大量实验证明，与以前的新方法相比，该算法在复杂度和信号保真度方面对各种图上的信号都获得了有竞争力的性能增益。
Analyzing and Improving the Robustness of Tabular Classifiers using Counterfactual Explanations
Authors: Peyman Rasouli, Ingrid Chieh Yu
Date: 2021-12-13 00:00:00
Proceeding: 20th IEEE International Conference on Machine Learning and Applications (ICMLA) 2021 12
Tasks: Adversarial Robustness, counterfactual, Counterfactual Explanation, Explainable artificial intelligence

Recent studies have revealed that Machine Learning (ML) models are vulnerable to adversarial perturbations. Such perturbations can be intentionally or accidentally added to the original inputs, evading the classifier's behavior to misclassify the crafted samples. A widely-used solution is to retrain the model using data points generated by various attack strategies. However, this creates a classifier robust to some particular evasions and can not defend unknown or universal perturbations. Counterfactual explanations are a specific class of post-hoc explanation methods that provide minimal modification to the input features in order to obtain a particular outcome from the model. In addition to the resemblance of counterfactual explanations to the universal perturbations, the possibility of generating instances from specific classes makes such approaches suitable for analyzing and improving the model's robustness. Rather than explaining the model's decisions in the deployment phase, we utilize the distance information obtained from counterfactuals and propose novel metrics to analyze the robustness of tabular classifiers. Further, we introduce a decision boundary modification approach using customized counterfactual data points to improve the robustness of the models without compromising their accuracy. Our framework addresses the robustness of black-box classifiers in the tabular setting, which is considered an under-explored research area. Through several experiments and evaluations, we demonstrate the efficacy of our approach in analyzing and improving the robustness of black-box tabular classifiers.
Explainable Debugger for Black-box Machine Learning Models
Authors: Peyman Rasouli, Ingrid Chieh Yu
Date: 2021-07-18 00:00:00
Proceeding: 2021 International Joint Conference on Neural Networks (IJCNN) 2021 7
Tasks: Anomaly Detection, BIG-bench Machine Learning, Data Visualization, Decision Making, Explainable artificial intelligence, Explanation Generation, Feature Importance

The research around developing methods for debugging and refining Machine Learning (ML) models is still in its infancy. We believe employing tailored tools in the development process can help developers in creating more trustworthy and reliable models. This is particularly essential for creating black-box models such as deep neural networks and random forests, as their opaque decision-making and complex structure prevent detailed investigations. Although many explanation techniques provide interpretability in terms of predictive features for a mispredicted instance, it would be beneficial for a developer to find a partition of the training data that significantly influences the anomaly. Such responsible partitions can be subjected to data visualization and data engineering in the development phase to improve the model's accuracy. In this paper, we propose a systematic debugging framework for the development of ML models that guides the data engineering process using the model's decision boundary. Our approach finds the influential neighborhood of anomalous data points using observation-level feature importance and explains them via a novel quasi-global explanation technique. It is also equipped with a robust global explanation approach to reveal general trends and expose potential biases in the neighborhoods. We demonstrate the efficacy of the devised framework through several experiments on standard data sets and black-box models and propose various guidelines on how the framework's components can be practically useful from a developer's perspective.
EXPLAN: Explaining Black-box Classifiers using Adaptive Neighborhood Generation
Authors: Peyman Rasouli, Ingrid Chieh Yu
Date: 2020-07-19 00:00:00
Proceeding: 2020 International Joint Conference on Neural Networks (IJCNN) 2020 7
Tasks: Explainable artificial intelligence, Explainable Models, Explanation Fidelity Evaluation

Defining a representative locality is an urgent challenge in perturbation-based explanation methods, which influences the fidelity and soundness of explanations. We address this issue by proposing a robust and intuitive approach for EXPLaining black-box classifiers using Adaptive Neighborhood generation (EXPLAN). EXPLAN is a module-based algorithm consisted of dense data generation, representative data selection, data balancing, and rule-based interpretable model. It takes into account the adjacency information derived from the black-box decision function and the structure of the data for creating a representative neighborhood for the instance being explained. As a local model-agnostic explanation method, EXPLAN generates explanations in the form of logical rules that are highly interpretable and well-suited for qualitative analysis of the model's behavior. We discuss fidelity-interpretability trade-offs and demonstrate the performance of the proposed algorithm by a comprehensive comparison with state-of-the-art explanation methods LIME, LORE, and Anchor. The conducted experiments on real-world data sets show our method achieves solid empirical results in terms of fidelity, precision, and stability of explanations.
Meaningful Data Sampling for a Faithful Local Explanation Method
Authors: Peyman Rasouli, Ingrid Chieh Yu
Date: 2019-10-18 00:00:00
Proceeding: 20th International Conference on Intelligent Data Engineering and Automated Learning (IDEAL), Lecture Notes in Computer Science 2019 10
Tasks: Explainable artificial intelligence, Explanation Fidelity Evaluation, Feature Importance

Data sampling has an important role in the majority of local explanation methods. Generating neighborhood samples using either the Gaussian distribution or the distribution of training data is a widely-used procedure in the tabular data case. Generally, this approach has several weaknesses: first, it produces a uniform data which may not represent the actual distribution of samples; second, disregarding the interaction between features tends to create unlikely samples; and third, it may fail to define a compact and diverse locality for the sample being explained. In this paper, we propose a sampling methodology based on observation-level feature importance to derive more meaningful perturbed samples. To evaluate the efficiency of the proposed approach we applied it to the LIME explanation method. The conducted experiments demonstrate considerable improvements in terms of fidelity and explainability.