Reviewerly/exHarmony huggingface dataset:
A Transductive Forest for Anomaly Detection with Few Labels

Extensive labeled training data for anomaly detection is enormously expensive and often unavailable in data-sensitive applications due to privacy constraints. We propose TransForest, a transductive forest for anomaly detection, in the semi-supervised setting where few labels are available. Guided by little label information, TransForest pushes classification boundaries toward sensitive areas where abnormal and normal points are located, increasing learning capacity. Empirically, TransForest is competitive with other unsupervised and semi-supervised representative detectors given a small number of labeled points. TransForest also offers a feature importance ranking consistent with the rankings provided by popular supervised forests on low-dimensional data sets. Our code is available at https://github.com/jzha968/transForest .
Transformer-Based Contrastive Multi-view Clustering via Ensembles

Multi-view spectral clustering has achieved considerable performance in practice because of its ability to explore nonlinear structure information. However, most existing methods belong to shallow models and are sensitive to the original similarity graphs. In this work, we proposed a novel model of Transformer-based contrastive multi-view clustering via ensembles (TCMCE) to solve the above issues. Our model integrates the self-attention mechanism, ensemble clustering, graph reconstruction, and contrastive learning into a unified framework. From the viewpoint of orthogonal and nonnegative graph reconstruction, TCMCE aims to learn a common spectral embedding as the indicator matrix. Then the graph contrastive learning is performed on the reconstructed graph based on the fusion graph via ensembles. Extensive experiments on six real-world datasets have verified the effectiveness of our model on multi-view clustering tasks compared with the state-of-the-art models.
Decompose, Then Reconstruct: A Framework of Network Structures for Click-Through Rate Prediction

Feature interaction networks are crucial for click-through rate (CTR) prediction in many applications. Extensive studies have been conducted to boost CTR accuracy by constructing effective structures of models. However, the performance of feature interaction networks is greatly influenced by the prior assumptions made by the model designer regarding its structure. Furthermore, the structures of models are highly interdependent, and launching models in different scenarios can be arduous and time-consuming. To address these limitations, we introduce a novel framework called DTR, which redefines the CTR feature interaction paradigm from a new perspective, allowing for the decoupling of its structure. Specifically, DTR first decomposes these models into individual structures and then reconstructs them within a unified model structure space, consisting of three stages: Mask, Kernel, and Compression. Each stage of DTR’s exploration of a range of structures is guided by the characteristics of the dataset or the scenario. Theoretically, we prove that the structure space of DTR not only incorporates a wide range of state-of-the-art models but also provides potentials to identify better models. Experiments on two public real-world datasets demonstrate the superiority of DTR, which outperforms state-of-the-art models.
k-SubMix: Common Subspace Clustering on Mixed-Type Data

Clustering heterogeneous data is an ongoing challenge in the data mining community. The most prevalent clustering methods are designed to process datasets with numerical features only, but often datasets consist of mixed numerical and categorical features. This requires new approaches capable of handling both kinds of data types. Further, the most relevant cluster structures are often hidden in only a few features. Thus, another key challenge is to detect those specific features automatically and abandon features not relevant for clustering. This paper proposes the subspace mixed-type clustering algorithm k-SubMix, which tackles both challenges. Its cost function can handle both numerical and categorical features while simultaneously identifying those with the biggest impact for a high-quality clustering result. Unlike other subspace mixed-type clustering methods, k-SubMix preserves inter-cluster comparability, as it is the first mixed-type approach that defines a common subspace for all clusters. Extensive experiments show that k-SubMix outperforms competitive methods and reduces the data’s complexity by a simultaneous dimensionality reduction.
Contrastive Learning with Cluster-Preserving Augmentation for Attributed Graph Clustering

Graph contrastive learning has attracted considerable attention and made remarkable progress in node representation learning and clustering for attributed graphs. However, existing contrastive-based clustering methods separate the processes of node representation learning and graph clustering into two stages, making it difficult to ensure good clustering. Therefore, it remains a challenge to design an effective contrastive learning method that jointly optimizes node representations and graph clustering. Moreover, existing random augmentation strategies to generate contrastive views may destroy the original topological structures of clusters in graphs. So it is crucial to construct an augmented graph that preserves the cluster structure of a given graph while benefitting graph clustering. To address these problems, we propose a contrastive learning method with cluster-preserving augmentation for attributed graph clustering. Specifically, we construct a contrasting view based on the generated kNN graph and edge betweenness centrality to preserve the inherent cluster structure of a graph. Then, a multilevel contrastive mechanism is proposed to maximize the agreement between node representations in multiple latent spaces. Finally, the objective of node representation learning is jointly optimized with the self-supervised clustering objective to obtain cluster distributions and discriminative node representations simultaneously. Extensive experiments on seven widely used real-world graphs demonstrate that the proposed model consistently outperforms existing state-of-the-art methods on clustering tasks.
ActiveGLAE: A Benchmark for Deep Active Learning with Transformers

Deep active learning (DAL) seeks to reduce annotation costs by enabling the model to actively query instance annotations from which it expects to learn the most. Despite extensive research, there is currently no standardized evaluation protocol for transformer-based language models in the field of DAL. Diverse experimental settings lead to difficulties in comparing research and deriving recommendations for practitioners. To tackle this challenge, we propose the ActiveGLAE benchmark, a comprehensive collection of data sets and evaluation guidelines for assessing DAL. Our benchmark aims to facilitate and streamline the evaluation process of novel DAL strategies. Additionally, we provide an extensive overview of current practice in DAL with transformer-based language models. We identify three key challenges - data set selection, model training, and DAL settings - that pose difficulties in comparing query strategies. We establish baseline results through an extensive set of experiments as a reference point for evaluating future work. Based on our findings, we provide guidelines for researchers and practitioners.
DSV: An Alignment Validation Loss for Self-supervised Outlier Model Selection

Self-supervised learning (SSL) has proven effective in solving various problems by generating internal supervisory signals. Unsupervised anomaly detection, which faces the high cost of obtaining true labels, is an area that can greatly benefit from SSL. However, recent literature suggests that tuning the hyperparameters (HP) of data augmentation functions is crucial to the success of SSL-based anomaly detection (SSAD), yet a systematic method for doing so remains unknown. In this work, we propose DSV (Discordance and Separability Validation), an unsupervised validation loss to select high-performing detection models with effective augmentation HPs. DSV captures the alignment between an augmentation function and the anomaly-generating mechanism with surrogate losses, which approximate the discordance and separability of test data, respectively. As a result, the evaluation via DSV leads to selecting an effective SSAD model exhibiting better alignment, which results in high detection accuracy. We theoretically derive the degree of approximation conducted by the surrogate losses and empirically show that DSV outperforms a wide range of baselines on 21 real-world tasks.
cuSLINK: Single-Linkage Agglomerative Clustering on the GPU

In this paper, we propose cuSLINK, a novel and state-of-the-art reformulation of the SLINK algorithm on the GPU which requires only O(Nk) space and uses a parameter k to trade off space and time. We also propose a set of novel and reusable building blocks that compose cuSLINK. These building blocks include highly optimized computational patterns for k-NN graph construction, spanning trees, and dendrogram cluster extraction. We show how we used our primitives to implement cuSLINK end-to-end on the GPU, further enabling a wide range of real-world data mining and machine learning applications that were once intractable. In addition to being a primary computational bottleneck in the popular HDBSCAN algorithm, the impact of our end-to-end cuSLINK algorithm spans a large range of important applications, including cluster analysis in social and computer networks, natural language processing, and computer vision.
Real: A Representative Error-Driven Approach for Active Learning

Given a limited labeling budget, active learning (al) aims to sample the most informative instances from an unlabeled pool to acquire labels for subsequent model training. To achieve this, al typically measures the informativeness of unlabeled instances based on uncertainty and diversity. However, it does not consider erroneous instances with their neighborhood error density, which have great potential to improve the model performance. To address this limitation, we propose Real, a novel approach to select data instances with Representative Errors for Active Learning. It identifies minority predictions as pseudo errors within a cluster and allocates an adaptive sampling budget for the cluster based on estimated error density. Extensive experiments on five text classification datasets demonstrate that Real consistently outperforms all best-performing baselines regarding accuracy and F1-macro scores across a wide range of hyperparameter settings. Our analysis also shows that Real selects the most representative pseudo errors that match the distribution of ground-truth errors along the decision boundary. Our code is publicly available at https://github.com/withchencheng/ECML_PKDD_23_Real .
Quantifying Robustness to Adversarial Word Substitutions

Deep-learning-based NLP models are found to be vulnerable to word substitution perturbations. Before they are widely adopted, the fundamental issues of robustness need to be addressed. Along this line, we propose a formal framework to evaluate word-level robustness. First, to study safe regions for a model, we introduce robustness radius which is the boundary where the model can resist any perturbation. As calculating the maximum robustness radius is computationally hard, we estimate its upper and lower bound. We repurpose attack methods as ways of seeking an upper bound and design a pseudo-dynamic programming algorithm for a tighter upper bound. Then verification method is utilized for a lower bound. Further, for evaluating the robustness of regions outside a safe radius, we reexamine robustness from another view: quantification. A robustness metric with a rigorous statistical guarantee is introduced to measure the quantification of adversarial examples, which indicates the model’s susceptibility to perturbations outside the safe radius. The metric helps us figure out why state-of-the-art models like BERT can be easily fooled by a few word substitutions, but generalize well in the presence of real-world noises.
A Baseline Generative Probabilistic Model for Weakly Supervised Learning

Finding relevant and high-quality datasets to train machine learning models is a major bottleneck for practitioners. Furthermore, to address ambitious real-world use-cases there is usually the requirement that the data come labelled with high-quality annotations that can facilitate the training of a supervised model. Manually labelling data with high-quality labels is generally a time-consuming and challenging task and often this turns out to be the bottleneck in a machine learning project. Weakly Supervised Learning (WSL) approaches have been developed to alleviate the annotation burden by offering an automatic way of assigning approximate labels (pseudo-labels) to unlabelled data based on heuristics, distant supervision and knowledge bases. We apply probabilistic generative latent variable models (PLVMs), trained on heuristic labelling representations of the original dataset, as an accurate, fast and cost-effective way to generate pseudo-labels. We show that the PLVMs achieve state-of-the-art performance across four datasets. For example, they achieve 22% points higher F1 score than Snorkel in the class-imbalanced Spouse dataset. PLVMs are plug-and-playable and are a drop-in replacement to existing WSL frameworks (e.g. Snorkel) or they can be used as baseline high-performance models for more complicated algorithms, giving practitioners a compelling accuracy boost.
Deployment Strategies for the Global Coverage of Quantum Key Distribution Network

We present a comprehensive literature review and comparative study on the deployment strategies of quantum key distribution (QKD) networks for global coverage. The state-of-the-art deployment strategies, including terrestrial QKD via optical fibers, free-space QKD via ground-based fixed links and ground-to-air dynamic links, as well as satellite QKD, are reviewed and compared in terms of channel loss, interference, distance limit, connection topology, and deployment cost. Selection criteria and deployment strategies are developed to enable a global coverage of QKD networks from intercontinental, long-haul to metro, and access networks.&#x0D;
Class-Conditional Label Noise in Astroparticle Physics

Class-conditional label noise characterizes classification tasks in which the training set labels are randomly flipped versions of the actual ground-truth. The analysis of telescope data in astroparticle physics poses this problem with a novel condition: one of the class-wise label flip probabilities is known while the other is not. We address this condition with an objective function for optimizing the decision thresholds of existing classifiers. Our experiments on several imbalanced data sets demonstrate that accounting for the known label flip probability substantially improves the learning outcome over existing methods for learning under class-conditional label noise. In astroparticle physics, our proposal achieves an improvement in predictive performance and a considerable reduction in computational requirements. These achievements are a direct result of our proposal’s ability to learn from real telescope data, instead of relying on simulated data as is common practice in the field.
Rectifying Bias in Ordinal Observational Data Using Unimodal Label Smoothing

This paper proposes a novel approach for modeling observational data in the form of expert ratings, which are commonly given on an ordered (numerical or ordinal) scale. In practice, such ratings are often biased, due to the expert’s preferences, psychological effects, etc. Our approach aims to rectify these biases, thereby preventing machine learning methods from transferring them to models trained on the data. To this end, we make use of so-called label smoothing, which allows for redistributing probability mass from the originally observed rating to other ratings, which are considered as possible corrections. This enables the incorporation of domain knowledge into the standard cross-entropy loss and leads to flexibly configurable models. Concretely, our method is realized for ordinal ratings and allows for arbitrary unimodal smoothings using a binary smoothing relation. Additionally, the paper suggests two practically motivated smoothing heuristics to address common biases in observational data, a time-based smoothing to handle concept drift and a class-wise smoothing based on class priors to mitigate data imbalance. The effectiveness of the proposed methods is demonstrated on four real-world goodwill assessment data sets of a car manufacturer with the aim of automating goodwill decisions. Overall, this paper presents a promising approach for modeling ordinal observational data that can improve decision-making processes and reduce reliance on human expertise.
Contrastive Learning-Based Imputation-Prediction Networks for In-hospital Mortality Risk Modeling Using EHRs

Predicting the risk of in-hospital mortality from electronic health records (EHRs) has received considerable attention. Such predictions will provide early warning of a patient’s health condition to healthcare professionals so that timely interventions can be taken. This prediction task is challenging since EHR data are intrinsically irregular, with not only many missing values but also varying time intervals between medical records. Existing approaches focus on exploiting the variable correlations in patient medical records to impute missing values and establishing time-decay mechanisms to deal with such irregularity. This paper presents a novel contrastive learning-based imputation-prediction network for predicting in-hospital mortality risks using EHR data. Our approach introduces graph analysis-based patient stratification modeling in the imputation process to group similar patients. This allows information of similar patients only to be used, in addition to personal contextual information, for missing value imputation. Moreover, our approach can integrate contrastive learning into the proposed network architecture to enhance patient representation learning and predictive performance on the classification task. Experiments on two real-world EHR datasets show that our approach outperforms the state-of-the-art approaches in both imputation and prediction tasks.
Weak Supervision and Clustering-Based Sample Selection for Clinical Named Entity Recognition

Abstract One of the central tasks of medical text analysis is to extract and structure meaningful information from plain-text clinical documents. Named Entity Recognition (NER) is a sub-task of information extraction that involves identifying predefined entities from unstructured free text. Notably, NER models require large amounts of human-labeled data to train, but human annotation is costly and laborious and often requires medical training. Here, we aim to overcome the shortage of manually annotated data by introducing a training scheme for NER models that uses an existing medical ontology to assign weak labels to entities and provides enhanced domain-specific model adaptation with in-domain continual pretraining. Due to limited human annotation resources, we develop a specific module to collect a more representative test dataset from the data lake than a random selection. To validate our framework, we invite clinicians to annotate the test set. In this way, we construct two Finnish medical NER datasets based on clinical records retrieved from a hospital’s data lake and evaluate the effectiveness of the proposed methods. The code is available at https://github.com/VRCMF/HAM-net.git .
Aspect-Based Complaint and Cause Detection: A Multimodal Generative Framework with External Knowledge Infusion

Customer reviews often contain valuable feedback about a product or service, but it can be challenging to extract specific complaints and their underlying causes from the text. Despite the use of various methods to detect and analyze complaints, no studies have concentrated on thoroughly examining complaints at the aspect-level and the underlying reasons for such aspect-level complaints. We add the rationale annotation for the aspect-based complaint classes in a publicly available benchmark multimodal complaint dataset (CESAMARD), which spans five domains (books, electronics, edibles, fashion, and miscellaneous). Current multimodal complaint detection methods treat these tasks as classification problems and do not utilize external knowledge. The present study aims to tackle these concerns. We propose a knowledge-infused unified Multimodal Generative framework for Aspect-based complaint and Cause detection (MuGACD) by reframing the multitasking problem as a multimodal text-to-text generation task. Our proposed methodology established a benchmark performance in the novel aspect-based complaint and cause detection task based on extensive evaluation. We also demonstrated that our model consistently outperformed all other baselines and state-of-the-art models in both full and few-shot settings (The dataset and code are available at https://github.com/Raghav10j/ECML23 ).
Context-Aware Deep Time-Series Decomposition for Anomaly Detection in Businesses

Detecting anomalies in time series has become increasingly challenging as data collection technology develops, especially in real-world communication services, which require contextual information for precise prediction. To address this challenge, researchers usually use time-series decomposition to reveal underlying patterns, e.g., trends and seasonality. However, existing decomposition-based anomaly detectors do not explicitly consider such contextual information, limiting their ability to correctly detect contextual cases. This paper proposes Time-CAD, a new context-aware deep time-series decomposition framework to detect anomalies for a more practical scenario in real-world businesses. We verify the effectiveness of the novel design for integrating contextual information into deep time-series decomposition through extensive experiments on four real-world benchmarks, demonstrating improvements of up to $$46\%$$ in time-series aware $$F_1$$ score on average.
Solar irradiance prediction using reinforcement learning pre-trained with limited historical data

Accurate day-ahead forecasting of solar irradiance is crucial for maintaining a steady power supply and minimizing energy losses. To date, various solar irradiance prediction models have been established, but these typically require extensive weather data collected over long periods within the area of prediction or consistent updates using field measurements. This research introduces a reinforcement learning-based model capable of long-term solar irradiance prediction, even in areas with limited accumulated data. Our proposed model can forecast solar radiation for more than a year using just two weeks of solar radiation learning and readily available weather forecasts. It demonstrated a promising performance, with an annual average CVRMSE error of 7.0%, which is a more optimized predictive performance than the 12.8% CVRMSE yielded by the existing LSTM-based Reference model constructed by adding out-of-atmosphere solar radiation input values.