Reviewerly/exHarmony huggingface dataset:
Sequential Anomaly Detection with Local and Global Explanations

Sequential anomaly detection has been studied for decades because of its wide spectrum of applications and obtained significant improvement in recent years by utilizing deep learning techniques. As an increasing number of anomaly detection models are applied to high-stake tasks involving human beings, it is critical to understand the reasons why the samples are labeled as anomalies. In this work, we propose a Globally and Locally Explainable Anomaly Detection (GLEAD) framework targeting sequential data. Especially, considering that the anomalies are usually diverse, we make use of the multi-head self-attention techniques to derive representations for sequences as well as prototypes, which capture a variety of patterns in anomalies. The attention mechanism highlights the abnormal entries with high attention weights in the abnormal sequences for the local explanation. Moreover, the prototypes of anomalies encoding the common patterns of abnormal sequences are derived to achieve the global explanation. Experimental results on two sequential anomaly detection datasets show that our approach can detect abnormal sequences and provide local and global explanations.
Anomaly Detection in Cybersecurity Events Through Graph Neural Network and Transformer Based Model: A Case Study with BETH Dataset

With the increasing prevalence of the internet, detecting malicious behavior is becoming a greater need. This problem can be formulated as an anomaly detection task on provenance data, where attacks are detectable as anomalies in the behavior of the system. While network data is quite prevalent, we focus on system logs and propose a novel approach with two main components. The first is to make use of the graph-like structure of the logs in which processes enact events and generate additional processes, using a graph neural network (GNN) to produce representations of each event which encode information about their neighboring events in an unsupervised manner. The second is to make use of the complex features such as command arguments which vary widely and cannot be used in the presented format as features in typical machine learning algorithms. If these features are instead encoded using transformer models, they can then be used in other algorithms such as a GNN or anomaly detector. These two approaches combined improve anomaly detection results for the BETH dataset by around 8 percent as compared to the manually engineered features alone.
Lossy Predictive Models for Accurate Classification Algorithms

Recent years have witnessed an upsurge of interest in lossy compression due to its potential to significantly reduce data volume with adequate exploitation of the spatiotemporal properties of IoT datasets. However, striking a balance between compression ratios and data fidelity is challenging, particularly when losing data fidelity impacts downstream data analytics noticeably. In this paper, we propose a lossy prediction model dealing with binary classification analytics tasks to minimize the impact of the error introduced due to lossy compression. We specifically focus on five classification algorithms for frost prediction in agricultural fields allowing preparation by the predictive advisories to provide helpful information for timely services. While our experimental evaluations reaffirm the nature of lossy compressions where allowing higher errors offers higher compression ratios, we also observe that the classification performance in terms of accuracy and F-1 score differs among all the algorithms we evaluated. Specifically, random forest is the best lossy prediction model for classifying frost. Lastly, we show the robustness of the lossy prediction model based on the data fidelity in prediction performance.
An Efficient In-Memory Analytics System Based on Persistent Memory

With the development of big data applications, online analytics systems that aim to offer decision support for various businesses and tasks have become a research focus, which calls for efficient approaches to handling OLAP queries. However, traditional OLAP systems suffer from the costly interactions with disks or SSDs, making them hard to deliver high performance for OLAP query processing. In this paper, we propose to use the emerging persistent memory to construct an efficient in-memory analytics system to improve the performance for OLAP query processing. We present the overall architecture as well as the detailed algorithms for the proposed system, which is named PM-Picker, and finally discuss the implementation issue of the system.
Explainable Machine Learning Models for Evaluating Government Grantmaking

This study compares two machine learning algorithms, decision trees and k-nearest neighbor, on the effectiveness of predicting success for organizations applying for arts and culture-related grants. Previous work in predictive modeling for grant applications found that accurate predictions were possible but were unable to provide explanations for model performance. Additionally, the data used in prior studies focused primarily on applicant academic pedigree and grant application history rather than details of grant application materials. Our research remedies this issue by training "white-box" models on application data from arts and cultural organizations applying to a government grant program in Chicago. Our results found that decision trees proved more accurate than k-nearest neighbors in predicting grant application success. Additionally, because the decision tree carries with it a ranking of the key factors for success, our approach provides useful information that may be beneficial to government agencies in improving the decision process and for applicants in evaluating past grant programs and predicting the success of future applications.
Dynamic Graph Summarization: Optimal and Scalable

Dynamic graph summarization is the task of obtaining and updating a summary of the current snapshot of a dynamic graph when changes (edge insertions/deletions) occur in the graph. As real graphs are massive and undergoing lots of changes, we need dynamic summarization algorithms that scale and are able to respond rapidly to changes in the graph. In this paper, we present two algorithms for lossless summarization of dynamic graphs. We first give an algorithm (Optimal) that is able to obtain and dynamically update the smallest-possible-anytime lossless summary in terms of node reduction. We achieve up to 8 orders of magnitude running time improvement over batch counterparts, and up to 12x improvement over the state-of-art in dynamic graph summarization, while at the same time offering up to 6x improvement in node reduction. We then present an even faster lossless summarization algorithm (Scalable), which goes further into speeding up dynamic updates by offering an additional order of magnitude improvement over Optimal at the cost of having lesser node reduction. Extensive experiments show that Scalable offers node reduction rates that are close to those of Optimal for many datasets. As such, Scalable is a preferred choice when speed of change is very high.
Permutation-Invariant Tabular Data Synthesis

Tabular data synthesis is an emerging approach to circumvent strict regulations on data privacy while discovering knowledge through big data. Although state-of-the-art AI-based tabular data synthesizers, e.g., table-GAN, CTGAN, TVAE, and CTAB-GAN, are effective at generating synthetic tabular data, their training is sensitive to column permutations of input data. In this paper, we first c onduct a n e xtensive e mpirical s tudy to disclose such a property of permutation invariance and an in-depth analysis of the existing synthesizers. We show that changing the input column order worsens the statistical difference between real and synthetic data by up to 38.67% due to the encoding of tabular data and the network architectures. To fully unleash the potential of big synthetic tabular data, we propose two solutions: (i) AE-GAN, a synthesizer that uses an autoencoder network to represent the tabular data and GAN networks to synthesize the latent representation, and (ii) a feature sorting algorithm to find t he s uitable c olumn o rder o f i nput d ata f or CNN-based synthesizers. We evaluate the proposed solutions on five datasets in terms of the sensitivity to the column permutation, the quality of synthetic data, and the utility in downstream analyses. Our results show that we enhance the property of permutation-invariance when training synthesizers and further improve the quality and utility of synthetic data, up to 22%, compared to the existing synthesizers.
Twitter Accounts Suggestion: Pipeline Technique SpaCy Entity Recognition

Twitter Accounts Suggestion is concerned with recommending accounts for the users according to their tweet contents. Twitter contains a massive amount of data that can be useful for knowing each user’s preferences. This paper uses Named Entity Recognition (NER), one of the techniques used in Natural Language Processing (NLP). We propose a pipeline technique to analyze the textual content of tweets to recommend the proper user accounts (people or organizations) to facilitate mentioning other user accounts in the tweets.
An LSTM Encoder-Decoder Approach for Unsupervised Online Anomaly Detection in Machine Learning Packages for Streaming Data

Anomalous behavior detection is an important component in many applications. Anomalies can represent problematic situations where early detection is critical to make situational assessments in the event of unexpected conditions. For many problems, the state of the art in machine learning is batch learning. However, online anomaly detection algorithms have emerged as an alternative which offer rapid access to useful insights with fewer computing capacity requirements. Ubiquitous and accelerated data streams have led to the development of several machine learning algorithms that favor adaptive learning. Real-time digital environments necessitate tackling specific data analysis challenges such as copious volumes of accelerated, infinite data streams and the phenomenon of concept drift. These Big Data challenges also represent real opportunities for improving a multitude of processes. In this paper, a novel thresholding approach for online anomaly detection based on the long short-term memory recurrent neural network encoder-decoder architecture is proposed. Comparative results in applying various advanced algorithms on 5 streaming datasets from the UCI repository and 3 synthetic datasets are discussed. All datasets contain anomaly ground truth information to evaluate the effectiveness of algorithms in detecting anomalous instances in streaming data.
A monitoring framework for deployed machine learning models with supply chain examples

Actively monitoring machine learning models during production operations helps ensure prediction quality and detection and remediation of unexpected or undesired conditions. Monitoring models already deployed in big data environments brings the additional challenges of adding monitoring in parallel to the existing modelling workflow and controlling resource requirements. In this paper, we describe (1) a framework for monitoring machine learning models; and, (2) its implementation for a big data supply chain application. We use our implementation to study drift in model features, predictions, and performance on three real data sets. We compare hypothesis test and information theoretic approaches to drift detection in features and predictions using the Kolmogorov-Smirnov distance and Bhattacharyya coefficient. Results showed that model performance was stable over the evaluation period. Features and predictions showed statistically significant drifts; however, these drifts were not linked to changes in model performance during the time of our study.
Secure Implementation of Artificial Intelligence Applications for Anti-Money Laundering using Confidential Computing

Money laundering not only facilitates the perpetration of dangerous and illegal activities it also damages the credibility and integrity of the global financial system and the financial institutions through whom money is laundered. Despite most financial institutions adhering to prevailing laws and regulations designed to prevent the practice of money laundering, it has been difficult to stop illicit activity using conventional methods. Hence, to combat money laundering, financial institutions are increasingly focused on the adoption of new technologies involving the use of artificial intelligence (AI) and machine learning (ML). One barrier to adoption of these new techniques for anti-money laundering (AML), however, is the need to maintain the confidentiality of the massive quantities of data required to train AI models, a financial data is the subject of regulatory controls and a target for cyber threat actors. In response to these challenges, this paper presents a secure and scalable architecture for AI implementation that uses confidential computing technology to provide complete end-to-end protection of sensitive financial data and the intellectual property of AML algorithm developers. Generative adversarial networks (GANs) are demonstrated using cloud infrastructure secured using Intel® Software Guard Extensions (Intel® SGX). The reported solution architecture can be adapted to support federated machine learning (FML), at scale, between mutually distrusting institutions, with independent control of data security at rest, in transit, and in use by individual data owners.
AE-smnsMLC: Multi-Label Classification with Semantic Matching and Negative Label Sampling for Product Attribute Value Extraction

Product attribute value extraction plays an important role for many real-world applications in e-Commerce such as product search and recommendation. Previous methods treat it as a sequence labeling task that needs more annotation for position of values in the product text. This limits their application to real-world scenario in which only attribute values are weakly-annotated for each product without their position. Moreover, these methods only use product text (i.e., product title and description) and do not consider the semantic connection between the multiple attribute values of a given product and its text, which can help attribute value extraction. In this paper, we reformulate this task as a multi-label classification task that can be applied for real-world scenario in which only annotation of attribute values is available to train models (i.e., annotation of positional information of attribute values is not available). We propose a classification model with semantic matching and negative label sampling for attribute value extraction. Semantic matching aims to capture semantic interactions between attribute values of a given product and its text. Negative label sampling aims to enhance the model’s ability of distinguishing similar values belonging to the same attribute. Experimental results on three subsets of a large real-world e-Commerce dataset demonstrate the effectiveness and superiority of our proposed model.
SandPiper: A Cost-Efficient Adaptive Framework for Online Recommender Systems

Online recommender systems have proven to have ubiquitous applications in various domains. To provide accurate recommendations in real time it is imperative to constantly train and deploy models with the latest data samples. This retraining involves adjusting the model weights by incorporating newly-arrived streaming data into the model to bridge the accuracy gap. To provision resources for the retraining, typically the compute is hosted on VMs, however, due to the dynamic nature of the data arrival patterns, stateless functions would be an ideal alternative over VMs, as they can instantaneously scale on demand. However, it is non-trivial to statically configure the stateless functions because the model retraining exhibits varying resource needs during different phases of retraining. Therefore, it is crucial to dynamically configure the functions to meet the resource requirements, while bridging the accuracy gap. In this paper, we propose Sandpiper, an adaptive framework that leverages stateless functions to deliver accurate predictions at low cost for online recommender systems. The three main ideas in Sandpiper are (i) we design a data-drift monitor that automatically triggers model retraining at required time intervals to bridge the accuracy gap due to incoming data drifts; (ii) we develop an online configuration model that selects the appropriate function configurations while maintaining the model serving accuracy within the latency and cost budget; and (iii) we propose a dynamic synchronization policy for stateless functions to speed up the distributed model retraining leading to cloud cost minimization. A prototype implementation on AWS shows that Sandpiper maintains the average accuracy above 90%, while 3.8× less expensive than the traditional VM-based schemes.
Differentially Private Federated Continual Learning with Heterogeneous Cohort Privacy

Differential privacy in federated learning has emerged as a promising solution for big data applications to achieve strong privacy guarantees. While prior work assumes that the privacy requirements are homogeneous across all clients, in practice, privacy requirements can differ across clients. In this paper, we introduce a cohort-based (ϵ, δ)-DP framework where privacy requirements and data distribution differ across these client cohorts. We show that the performance of existing differentially private stochastic algorithms degrade significantly for heterogeneous privacy scenarios, especially when the data is non-independent and identically distributed (non-iid). Moreover, we propose two novel continual learning-based DP training methods — DP-Synaptic intelligence (DP-SI) and DP-Rehearsal (DP-R) — to improve the model performance of cohorts with heterogeneous privacy budgets. We evaluate our approach on real datasets and show that our techniques outperform baseline techniques. Furthermore, our approach adapts to post-hoc privacy budget relaxations, providing greater flexibility in training models without significantly impacting performance.
A Semi-Supervised Learning Approach for Abnormal Event Prediction on Large Network Operation Time-Series Data

Large network logs, recording multivariate time series generated from heterogeneous devices and sensors in a network, can reveal important information about abnormal activities, such as network intrusions and packet losses. Existing machine learning methods for anomaly detection on multiple multivariate time series typically assume that 1) infrequent behaviors beyond some inference threshold are anomalous for unsupervised models or 2) require a large set of labeled normal and abnormal sequences for supervised models. However, in practice, the reported abnormal events might be available but incomplete and sparse (i.e., much fewer than normal cases). This paper presents a novel semi-supervised approach, SNetAD, that takes advantage of the incomplete and imbalanced labels to effectively learn separable feature embeddings of network activities representing normal and abnormal events. Specifically, SNetAD first generates network representations by capturing relationships across time points and between network devices. Then SNetAD encourages the embeddings to form two clusters using contrastive center loss and improves the separability of the learned clusters using labeled and unlabeled samples in a semi-supervised manner. The experiments demonstrate that SNetAD significantly outperforms state-of-the-art approaches for abnormal event prediction on a large real-world network log.
Time-aware Hyperbolic Graph Attention Network for Session-based Recommendation

Session-based Recommendation (SBR) is to predict users’ next interested items based on their previous browsing sessions. Existing methods model sessions as graphs or sequences to estimate user interests based on their interacted items to make recommendations. In recent years, graph-based methods have achieved outstanding performance on SBR. However, none of these methods consider temporal information, which is a crucial feature in SBR as it indicates timeliness or currency. Besides, the session graphs exhibit a hierarchical structure and are demonstrated to be suitable in hyperbolic geometry. But few papers design the models in hyperbolic spaces and this direction is still under exploration.In this paper, we propose Time-aware Hyperbolic Graph Attention Network (TA-HGAT) — a novel hyperbolic graph neural network framework to build a session-based recommendation model considering temporal information. More specifically, there are three components in TA-HGAT. First, a hyperbolic projection module transforms the item features into hyperbolic space. Second, the time-aware graph attention module models time intervals between items and the users’ current interests. Third, an evolutionary loss at the end of the model provides an accurate prediction of the recommended item based on the given timestamp. TA-HGAT is built in a hyperbolic space to learn the hierarchical structure of session graphs. Experimental results show that the proposed TA-HGAT has the best performance compared to ten baseline models on two real-world datasets.
Pattern Discovery with Utility Occupancy

To mine potential and helpful patterns, the majority of studies on pattern discovery from databases have been conducted in the last few decades. They have several obvious drawbacks: 1) Each thing stands out on its own and varies in significance based on factors including utility, risk, interest, and weight. 2) In specific application settings, an object has a favorable or unfavorable effect (e.g., products are often cross-sold and have positive or negative unit profits, which affect benefits). 3) The user could not have all the necessary information because frequent-based patterns typically only include a small percentage of the relevant patterns (for example, occupancy). To address this issue, we apply economic utility theory to the database and data mining fields. We provide a one-phase approach called pnHUO for discovering High Utility Occupancy patterns with positive and negative utility values that beyond frequency and usefulness. According to user interests, frequency, and utility occupancy, there are various utility occupancy patterns with positive and negative utility values. To hold the necessary data, a new frequency-utility tree and an indexed data structure called a positive-and-negative utility-occupancy list are created during the mining process. A number of pruning strategies are further developed using the determined upper bound of utility occupancy to reduce the search space. To evaluate the usefulness and efficiency of the suggested algorithm, five real datasets were tested in experiments, and the results were positive.
DBNex: Deep Belief Network and Explainable AI based Financial Fraud Detection

The majority of financial transactions are now conducted virtually around the world. The widespread use of credit cards and online transactions encourages fraudulent activity. Thus, one of the most demanding real-world challenges is fraud detection. Unbalanced datasets, in which there are a disproportionately high number of non-fraud samples compared to incidents of fraud, are one of the key obstacles to effective fraud detection. A further factor complicating the learning process for cutting-edge machine learning classifiers is how quickly fraud behaviour changes. Thus, in this study, we suggest an efficient fraud detection methodology. We propose a unique nonlinear embedded clustering to resolve imbalances in the dataset, followed by a Deep Belief Network for detecting fraudulent transactions. The proposed model achieved an accuracy of 94% with a 70:30 ratio of training-validation dataset.