Reviewerly/exHarmony huggingface dataset:
Continuous-Time Temporal Graph Learning on Provenance Graphs

Recent advances in Graph Neural Networks (GNNs) have matured the field of learning on graphs, making GNNs essential for prediction tasks in complex, interconnected, and evolving systems.In this paper, we focus on self-supervised, inductive learning for continuous-time dynamic graphs. Without compromising generality, we propose an approach to learn representations and mine anomalies in provenance graphs, which are a form of large-scale, heterogeneous, attributed, and continuous-time dynamic graphs used in the cybersecurity domain, syntactically resembling complex temporal knowledge graphs. We modify the Temporal Graph Network (TGN) framework to heterogeneous input data and directed edges, refining it specifically for inductive learning on provenance graphs. We present and release two pioneering large-scale, continuous-time temporal, heterogeneous, attributed benchmark graph datasets. The datasets incorporate expert-labeled anomalies, promoting subsequent research on representation learning and anomaly detection on intricate real-world networks. Comprehensive experimental analyses of modules, datasets, and baselines underscore the effectiveness of TGN-based inductive learning, affirming its practical utility in identifying semantically significant anomalies in real-world systems.
Modeling Uncertainty in Personalized Emotion Prediction with Normalizing Flows

Designing predictive models for subjective problems in natural language processing (NLP) remains challenging. This is mainly due to its non-deterministic nature and different perceptions of the content by different humans. It may be solved by Personalized Natural Language Processing (PNLP), where the model exploits additional information about the reader to make more accurate predictions. However, current approaches require complete information about the recipients to be straight embedded. Besides, the recent methods focus on deterministic inference or simple frequency-based estimations of the probabilities. In this work, we overcome this limitation by proposing a novel approach to capture the uncertainty of the forecast using conditional Normalizing Flows. This allows us to model complex multimodal distributions and to compare various models using negative log-likelihood (NLL). In addition, the new solution allows for various interpretations of possible reader perception thanks to the available sampling function. We validated our method on three challenging, subjective NLP tasks, including emotion recognition and hate speech. The comparative analysis of generalized and personalized approaches revealed that our personalized solutions significantly outperform the baseline and provide more precise uncertainty estimates. The impact on the text interpretability and uncertainty studies are presented as well. The information brought by the developed methods makes it possible to build hybrid models whose effectiveness surpasses classic solutions. In addition, an analysis and visualization of the probabilities of the given decisions for texts with high entropy of annotations and annotators with mixed views were carried out.
GNN-GMVO: Graph Neural Networks for Optimizing Gross Merchandise Value in Similar Item Recommendation

Similar item recommendation is a critical task in the e-Commerce industry, which helps customers explore similar and relevant alternatives based on their interested products. Despite the traditional machine learning models, Graph Neural Networks (GNNs), by design, can understand complex relations like similarity between products. However, in contrast to their wide usage in retrieval tasks and their focus on optimizing the relevance, the current GNN architectures are not tailored toward maximizing revenue-related objectives such as Gross Merchandise Value (GMV), which is one of the major business metrics for e-Commerce companies. In addition, defining accurate edge relations in GNNs is non-trivial in large-scale e-Commerce systems, due to the heterogeneity nature of the item-item relationships. This work aims to address these issues by designing a new GNN architecture called GNN-GMVO (Graph Neural Network - Gross Merchandise Value Optimizer). This model directly optimizes GMV while considering the complex relations between items. In addition, we propose a customized edge construction method to tailor the model toward similar item recommendation task and alleviate the noisy and complex item-item relations. In our comprehensive experiments on three real-world datasets, we show higher prediction performance and expected GMV for top ranked items recommended by our model when compared with selected state-of-the-art benchmark models.
Video Instance Segmentation Using Graph Matching Transformer

In this work, we study the challenge of video Instance Segmentation (VIS), which needs to track and segment multiple objects in videos automatically. We introduce a novel network from a graph matching perspective to formulate VIS, called GMP-VIS. Unlike traditional tracking-by-detection paradigm or bottom-up generative solutions, GMP-VIS uses a novel, learnable graph matching Transformer to predict the instances by heuristically learning the spatial-temporal relationships. Specifically, we take advantage of the powerful Transformer and exploit temporal feature aggregation to capture long-term temporal information across frames implicitly. After generating instance proposals for each frame, the difficult instance association problem is cast as a more leisurely, differentiable graph matching task. The graph matching mechanism performs the data association between current and historical frames based on the proposed instance feature, which can better infer the deformations and obscured foreground instances. Building graph-level annotation during network training allows our GMP-VIS to mine more structural supervision significantly distinguished from current VIS solutions. Our extensive experiments over three representative benchmarks, including YouTube-VIS <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">19</inf> , YouTubeVIS <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">21</inf> , and OVIS, demonstrate that GMP-VIS outperforms the current alternatives by a large margin. Code is availaible.
Defending Network IDS against Adversarial Examples with Continual Learning

Improving computer system security is one of the most critical issues in modern computer science. Machine learning algorithms increasingly support the construction of such solutions, mainly to detect network attacks. Such algorithms detect dangerous activity without requiring the manual formulation of expert rules to decide what activity constitutes an attack. Due to the emergence of new attacks, IDS (Intrusion Detection System) needs to be updated periodically. However, their update carried out without due care may also be exploited by an attacker who deliberately tries to influence the analyzed data to mislead the predictive model.The proposed approach uses adversarial examples to generate new network traffic patterns that are misclassified by the neural network inside the IDS. This approach could model the evolution of cyber threats to some extent and should allow for continuous improvement of the IDS. In this paper, we propose an original framework for simulating attacker-defender dynamics based on adversarial examples and show that it is possible to continuously improve IDS systems by applying continual learning strategies. The proposed approach has been tested based on experimental studies using known continual learning algorithms, and the experimental results confirm the usability of the proposed method.The results presented in this paper identify potential gaps in ML-based NIDS systems. At the same time, we show how these threats can be limited, which should contribute to mitigating some possible threads and the overall reliability of the intrusion detection process.
Resolving the Imbalance Issue in Hierarchical Disciplinary Topic Inference via LLM-based Data Augmentation

In addressing the imbalanced issue of data within the realm of Natural Language Processing, text data augmentation methods have emerged as pivotal solutions. This data imbalance is prevalent in the research proposals submitted during the funding application process. Such imbalances, resulting from the varying popularity of disciplines or the emergence of interdisciplinary studies, significantly impede the precision of downstream topic models that deduce the affiliated disciplines of these proposals. At the data level, proposals penned by experts and scientists are inherently complex technological texts, replete with intricate terminologies, which augmenting such specialized text data poses unique challenges. At the system level, this, in turn, compromises the fairness of AI-assisted reviewer assignment systems, which raises a spotlight on solving this issue. This study leverages large language models (Llama V1) as data generators to augment research proposals categorized within intricate disciplinary hierarchies, aiming to rectify data imbalances and enhance the equity of expert assignments. We first sample within the hierarchical structure to find the underrepresented class. Then we designed a prompt for keyword-based research proposal generation. Our experiments attests to the efficacy of the generated data, demonstrating that research proposals produced using the prompts can effectively address the aforementioned issues and generate high quality scientific text data, thus help the model overcome the imbalanced issue.
A Federated Multi-Modal Learning Framework Powered by Distributed Ledgers for Cyber-safe and Efficient UAV Delivery Systems

This paper develops a cutting-edge multimodal federated learning framework, integrated with distributed ledger technologies, designed specifically for UAV delivery scenarios. The framework adopts various data modalities, including user pictures, behavior, and location, to dynamically optimize delivery routes and schedules, thus enhancing both user privacy and security of the delivery process. By employing federated learning, this framework allows data to be processed locally on individual devices, significantly enhancing both user privacy and data integrity. The integration of distributed ledger technology ensures that all updates to the federated model are not only immutable and traceable, but also secure. Through comprehensive evaluations, our framework shows outstanding improvements in both the efficiency and security of UAV deliveries. These findings show the transformative potential of our approach to establish user-centric, efficient, and secured UAV delivery systems.
Learning and Adapting Diverse Representations for Cross-domain Few-shot Learning

Conventional few-shot learning (FSL) mainly focuses on transferring knowledge from a single source dataset to a recognition scenario with only a few training samples but still similar to the source domain. In this paper, we consider a more practical FSL setting where multiple semantically different datasets are available to address a wide range of FSL tasks, especially for some recognition scenarios beyond natural images, such as aerial and medical imagery. It can be referred to as multi-source cross-domain FSL. To tackle the problem, we propose a two-stage learning scheme, termed learning and adapting multi-source representations (LAMR). In the first stage, we propose a multi-head network to obtain efficient multi-domain representations, where all source domains share the same backbone except for the last parallel projection layers used for domain specialization. We train the representations in a multitask setting where each in-domain classification task is realized by a cosine classifier. In the second stage, considering that instance discrimination and class discrimination are crucial for robust recognition, we propose two contrastive objectives for adapting the pre-trained representations to be task-specialized on the few-shot data. Careful ablation studies verify that LAMR significantly improves representation transferability, showing consistent performance boosts for FSL. Experiments on the BSCD-FSL benchmarks demonstrate that LAMR can achieve state-of-the-art results, highlighting its versatility and effectiveness for FSL of both natural and specific imaging.
Cross-Modal Attention Network for Sign Language Translation

Context serves as a critical role in sign language translation (SLT), e.g., textual questions from the hearing people and visual answers from the hearing-impaired people in communication have a contextual relationship, which can facilitate the understanding of sign language. Current SLT approaches mainly focus on learning visual representations and translate it into spoken or written language sentences, while neglect the utilization of context information in question-answer (QA) scenarios. In this paper, we propose a novel and effective Cross-Modal Attention Network (CMA-Net) to better learn multimodal features (i.e., text and vision) for improving the translation accuracy of SLT with the aid of contextual information. Specifically, we design a cross-modal knowledge transfer module, which embeds the star-based attention mechanism into the Transformer model to explore a short-term and long-term interactive relationship between different modalities and achieve cross-modality message transfer. Besides, we propose a multi-task learning paradigm, i.e., using auxiliary recognition (sign2gloss) and translation (gloss2text) tasks to augment the main task with a pre-training strategy. Significant performance improvements on three public benchmark datasets demonstrate the effectiveness of CMA-Net and the usefulness of context information for SLT.
ICA model estimation using a mixed learning rule based on genetic algorithms and neural networks

This paper presents a method of estimating the independent component analysis model with a training algorithm based on a mixed learning algorithm of genetic algorithms with a neural network algorithm. The mixed training algorithm is applied to optimize the objective function negentropy used to estimate the ICA model. The proposed estimation algorithm improves the training scheme based on genetic algorithms by using for crossover the most suitable chromosomes evaluated by the objective function with the parameters calculated calculated accordingly by a multilayer neural network algorithm. The results produced by the proposed algorithm were compared with the results of applying standard methods for estimating independent components such as FastICA variants based on the standard Newton method or the secant method. The experimental results were established in blind source separation applications by using unidimensional and bidimensional signals.
SafeML: A Privacy-Preserving Byzantine-Robust Framework for Distributed Machine Learning Training

This paper introduces SafeML, a distributed machine learning framework that can address privacy and Byzantine robustness concerns during model training. It employs secret sharing and data masking techniques to secure all computations, while also utilizing computational redundancy and robust confirmation methods to prevent Byzantine nodes from negatively affecting model updates at each iteration of model training. The theoretical analysis and preliminary experimental results demonstrate the security and correctness of SafeML for mode training.
Efficient Distributed Core Graph Decomposition

Core decomposition is one of the most fundamental problems in graph analytics, which is associated with numerous applications, such as community detection, protein network analysis, and system structure analysis. As the sizes of graphs are becoming increasingly large, it is challenging to compute core decomposition on a single machine. In this paper, we study the problem of k-Core decomposition in the distributed environment. Specifically, we propose the distributed Filter-Array k-Core (FAkCore) algorithm, which adopts the commonly used Scatter-Gather framework. We design an auxiliary data structure of running counts for each vertex to track the statistics of its neighbors’ core number. It allows us to recompute the core number of a vertex only when the value is updated. Together with an enhanced message filtering mechanism, our method significantly reduces redundant computation and communication in the existing distributed k-Core decomposition algorithm. Experiments on 10 real-world graphs show that our method outperforms the baseline algorithms by 1.4 times on average and up to 2.2 times.
Explainable Artificial Intelligence for Interpretable Data Minimization

Black box models such as deep neural networks are increasingly being deployed in high-stakes fields, including justice, health, and finance. Furthermore, they require a huge amount of data, and such data often contains personal information. However, the principle of data minimization in the European Union’s General Data Protection Regulation requires collecting only the data that is essential to fulfilling a particular purpose. Implementing data minimization for black box models can be difficult because it involves identifying the minimum set of variables that are relevant to the model’s prediction, which may not be apparent without access to the model’s inner workings. In addition, users are often reluctant to share all their personal information. We propose an interactive system to reduce the amount of personal data by determining the minimal set of features required for a correct prediction using explainable artificial intelligence techniques. Our proposed method can inform the user whether the provided variables contain enough information for the model to make accurate predictions or if additional variables are necessary. This human-centered approach can enable providers to minimize the amount of personal data collected for analysis and may increase the user’s trust and acceptance of the system.
DropMix: Better Graph Contrastive Learning with Harder Negative Samples

While generating better negative samples for contrastive learning has been widely studied in the areas of CV and NLP, very few work has focused on graph-structured data. Recently, Mixup has been introduced to synthesize hard negative samples in graph contrastive learning (GCL). However, due to the unsupervised learning nature of GCL, without the help of soft labels, directly mixing representations of samples could inadvertently lead to the information loss of the original hard negative and further adversely affect the quality of the newly generated harder negative. To address the problem, in this paper, we propose a novel method DropMix to synthesize harder negative samples, which consists of two main steps. Specifically, we first select some hard negative samples by measuring their hardness from both local and global views in the graph simultaneously. After that, we mix hard negatives only on partial representation dimensions to generate harder ones and decrease the information loss caused by Mixup. We conduct extensive experiments to verify the effectiveness of DropMix on six benchmark datasets. Our results show that our method can lead to better GCL performance. Our data and codes are publicly available at https://github.com/Mayueq/DropMix-Code.
Asymmetric Polysemous Reasoning for Image-Text Matching

Image-text matching has received growing interest since it bridges vision and language. The key challenge lies in how to learn correspondence between image and text. Upon observation, we find existing works suffer from two limitations. Firstly, existing works use similar networks to extract features for different modalities without consider the modal characteristics, hinder the semantic reasoning in modalities. Secondly, existing works map a sentence into single space. However, due to the semantic ambiguity, such methods obviously cannot effectively deal with polysemy. In this paper, we present a new Asymmetric Polysemous Reasoning Network (APRN) to tailor for image and text modality characteristics. For image, APRN employs Graph Convolutional Network to implicitly learn the relationships between regions, and perform global semantic reasoning to get image representation. For text, given its ambiguity, we map text into different subspaces to learn distinct various meanings. Then we propose Selection Attention (SA) that combines image information to integrate the text features of subspaces, by selectively attending on the significant and representative subspaces and meanwhile casting aside the interferences of non-meaningful subspaces. Experiments validate that our method surpasses many recent state-of-the-arts with a clear margin for the image-text matching on Flickr30K and MS-COCO datasets.
ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphs

ChatGPT, as a recently launched large language model (LLM), has shown superior performance in various natural language processing (NLP) tasks. However, two major limitations hinder its potential applications: 1) the inflexibility of finetuning on downstream tasks, and 2) the lack of interpretability in the decision-making process. To tackle these limitations, we propose a novel framework that leverages the power of ChatGPT for specific tasks, such as text classification, while improving its interpretability. The proposed framework conducts a knowledge graph extraction task to extract refined and structural knowledge from the raw data using ChatGPT. The rich knowledge is then converted into a graph, which is further used to train an interpretable linear classifier to make predictions. To evaluate the effectiveness of our proposed method, we conduct experiments on four benchmark datasets. The results demonstrate that our method can significantly improve the prediction performance compared to directly utilizing ChatGPT for text classification tasks. Furthermore, our method provides a more transparent decision-making process compared with previous text classification methods. The code is available at https://github.com/sycny/ChatGraph.
Hard Anomaly Detection: An Adversarial Data Augmentation Solution

We propose a measure to divide anomaly detection into four categories. At the same time, we focus on a special and representative type of anomaly detection problem, called hard anomaly detection. Compared with ordinary anomaly detection where anomalies usually appear as outliers, the so-called hard anomaly has two characteristics in terms of the differences between abnormal and normal samples: 1) the boundary is exceedingly unobvious, and 2) the distribution is extremely imbalanced, both of which result in the difficulty of anomaly detection. It is really hard for regular classifiers to find the boundary between abnormal and normal samples here. To address the issue of hard anomaly detection, in this work, we first propose a quantitative definition of the hard anomaly according to detection difficulty and then design a dedicated solution framework, named Hard Anomaly Detection (HAD). Under the framework HAD, we devise a GAN-based (Generative Adversarial Network) method, called HadGAN, which produces both abnormal and normal samples with a similar distribution. We then pre-train a base anomaly detection model using the data generated by HadGAN. Moreover, we apply transfer learning to fine-tune the base model on real datasets. The superiority of our solution is demonstrated both by theoretical analysis as well as in extensive experiments.