bluuebunny/crossref_metadata_2025 huggingface dataset:
Partial Label Learning via Cost-Guided Retraining
Authors: Zhang Zhaoyuan , Liu Zhenbing , Lu Haoxiang
Published: October 2024

In partial label learning, each training sample corresponds to a set of candidate labels. The ground-truth label, hidden within this set, cannot be directly obtained during the training phase. The key to solving the partial label learning problem is to obtain ground-truth labels through label disambiguation. Existing works often rely on the label averaging assumption and do not fully investigate the class imbalance. Tail ground-truth labels are often overwhelmed by head pseudo-labels. The incorrectly identified labels could have contagiously negative impacts on the final predictions. In this paper, we propose a cost-guided retraining strategy, which achieves guidance and correction of disambiguation results, and provides instance-based class imbalance concerns for candidate labels. This approach significantly enhances the algorithm’s ability to handle class imbalance problems. The superiority of our method is demonstrated using 8 real-world datasets and 5 evaluation metrics. Code is available at https://github.com/DerrickZzyR/PL-CGR
Context Matters: Leveraging Spatiotemporal Metadata for Semi-Supervised Learning on Remote Sensing Images
Authors: Bernhard Maximilian , Hannan Tanveer , Strauß Niklas , Schubert Matthias
Published: October 2024

Remote sensing projects typically generate large amounts of imagery that can be used to train powerful deep neural networks. However, the amount of labeled images is often small, as remote sensing applications generally require expert labelers. Thus, semi-supervised learning (SSL), i.e., learning with a small pool of labeled and a larger pool of unlabeled data, is particularly useful in this domain. Current SSL approaches generate pseudo-labels from model predictions for unlabeled samples. As the quality of these pseudo-labels is crucial for performance, utilizing additional information to improve pseudo-label quality yields a promising direction. For remote sensing images, geolocation and recording time are generally available and provide a valuable source of information as semantic concepts, such as land cover, are highly dependent on spatiotemporal context, e.g., due to seasonal effects and vegetation zones. In this paper, we propose to exploit spatiotemporal metainformation in SSL to improve the quality of pseudo-labels and, therefore, the final model performance. We show that directly adding the available metadata to the input of the predictor at test time degenerates the prediction quality for metadata outside the spatiotemporal distribution of the training set. Thus, we propose a teacher-student SSL framework where only the teacher network uses metainformation to improve the quality of pseudo-labels on the training set. Correspondingly, our student network benefits from the improved pseudo-labels but does not receive metadata as input, making it invariant to spatiotemporal shifts at test time. Furthermore, we propose methods for encoding and injecting spatiotemporal information into the model and introduce a novel distillation mechanism to enhance the knowledge transfer between teacher and student. Our framework dubbed Spatiotemporal SSL can be easily combined with several state-of-the-art SSL methods, resulting in significant and consistent improvements on the BigEarthNet and EuroSAT benchmarks. Code is available at https://github.com/mxbh/spatiotemporal-ssl.
Link Prediction Without Learning
Authors: Delarue Simon , Bonald Thomas , Viard Tiphaine
Published: October 2024

Link prediction is a fundamental task in machine learning for graphs. Recently, Graph Neural Networks (GNNs) have gained in popularity and have become the default approach for solving this type of task. Despite the considerable interest for these methods, simple topological heuristics persistently emerge as competitive alternatives to GNNs. In this study, we show that this phenomenon is not an exception and that GNNs do not consistently establish a performance standard for link prediction on graphs. For this purpose, we identify several limitations in the current GNN evaluation methodology, such as the lack of variety in benchmark dataset characteristics and the limited use of diverse baselines outside of neural methods. In particular, we highlight that integrating feature information into topological heuristics remains a little-explored path. In line with this observation, we propose a simple non-neural model that leverages local structure, node feature, and graph feature information within a weighted combination. Experiments conducted on large variety of networks indicate that the proposed approach outperforms existing state-of-the-art GNNs and increases generalisation ability. Contrasting with GNNs, our approach does not rely on any learning process and therefore achieves superior results without sacrificing efficiency, showcasing a reduction of one to three orders of magnitude in computation time.
Distilling the Effects of Language Model Contamination
Authors: Mehrbakhsh Behzad , Martínez-Plumed Fernando , Hernández-Orallo José
Published: October 2024

The proportion of AI-generated content permeating the well of knowledge is increasing significantly. Large language models (LLMs) contribute to that contamination but they also suffer from it. However, it is yet to be clarified the effect of different sources of error, be it human-generated or LLM-generated. Controlling for the percentage of error, we explore the impact on LLM fine-tuning when errors come from humans, from other language models or are generated randomly using an aleatoric or epistemic source. In this paper, we compare these different types of error for in-distribution and out-of-distribution experimental settings. By analysing the levels of errors and their distribution, we find a nuanced view: while in-distribution human-generated noise seems more benign than the LLM-generated counterpart, in the out-of-distribution case the model-generated noise may not be necessarily worse.
R4: Reinforced Retriever-Reorder-Responder for Retrieval-Augmented Large Language Models
Authors: Zhang Taolin , Li Dongyang , Chen Qizhou , Wang Chengyu , Huang Longtao , Xue Hui , He Xiaofeng , Huang Jun
Published: October 2024

Retrieval-augmented large language models (LLMs) leverage relevant content retrieved by information retrieval systems to generate correct responses, aiming to alleviate the hallucination problem. However, existing retriever-responder methods typically append relevant documents to the prompt of LLMs to perform text generation tasks without considering the interaction of fine-grained structural semantics between the retrieved documents and the LLMs. This issue is particularly important for accurate response generation as LLMs tend to “lose in the middle” when dealing with input prompts augmented with lengthy documents. In this work, we propose a new pipeline named “Reinforced Retriever-Reorder-Responder” (R4) to learn document orderings for retrieval-augmented LLMs, thereby further enhancing their generation abilities while the large numbers of parameters of LLMs remain frozen. The reordering learning process is divided into two steps according to the quality of the generated responses: document order adjustment and document representation enhancement. Specifically, document order adjustment aims to organize retrieved document orderings into beginning, middle, and end positions based on graph attention learning, which maximizes the reinforced reward of response quality. Document representation enhancement further refines the representations of retrieved documents for responses of poor quality via document-level gradient adversarial learning. Extensive experiments demonstrate that our proposed pipeline achieves better factual question-answering performance on knowledge-intensive tasks compared to strong baselines across various public datasets. The source codes and trained models will be released upon paper acceptance.
Bipartite Time Series Network for Data Imputation
Authors: Amerise Ilaria Lucrezia , Fionda Valeria , Pirrò Giuseppe
Published: October 2024

The pervasive issue of missing data in statistical analysis and data science significantly impacts the integrity and accuracy of Time-Series Cross-Sectional (TSCS) datasets, extensively used in domains like health sciences and sensor applications. Traditional imputation methods are often inadequate for these datasets as they fail to address the complexities arising from cross-sectional and temporal data gaps. This paper introduces BiTSNet (Bipartite Time Series Network for Data Imputation), a novel approach designed to tackle the unique challenges of TSCS datasets by leveraging a dual representation of data. BiTSNet models cross-sectional time series data as sequences of bipartite graphs, where each graph represents a specific time step, and feature values are depicted as edge weights. Thus, missing values are interpreted as missing edge feature values, allowing BiTSNet to comprehensively capture spatial relationships within each time step and temporal relationships across steps. The framework uses Graph Neural Networks (GNNs) to process spatial dependencies within these bipartite graph representations and employs Recurrent Neural Networks (RNNs) to handle temporal dependencies. This integration enables BiTSNet to learn the intricate intra-time step relationships and the inter-time step dynamics effectively. Our approach not only preserves the longitudinal and cross-sectional integrity of the data but also ensures the production of valid and insightful conclusions from the enriched dataset. We evaluated BiTSNet on several datasets and compared it against the state-of-the-art approaches with encouraging results.
A Simple Yet Effective Interpretable Bayesian Personalized Ranking for Cognitive Diagnosis
Authors: Batel Arthur , Benouaret Idir , Fruitet Joan , Plantevit Marc , Robardet Céline
Published: October 2024

In the field of education, the automatic assessment of student profiles has become a crucial objective, driven by the rapid expansion of online tutoring systems and computerized adaptive testing. These technologies aim to democratize education and enhance student assessment by providing detailed insights into student profiles, which are essential for accurately predicting the outcomes of exercises, such as solving various types of mathematical equations. We aim to develop a model capable of predicting responses to a large set of questions within the Multi-Target Prediction framework while ensuring that this model is explainable, allowing us to quantify student performance in specific knowledge areas. Existing cognitive diagnosis algorithms often struggle to meet the dual requirement of accurately predicting exercise outcomes and maintaining interpretability. To address this challenge, we propose an alternative to the complexity of current advanced machine learning models. Instead, we introduce a direct yet highly effective Bayesian Personalized Ranking algorithm, called CD-BPR, which incorporates interpretability as a core learning objective. Extensive experiments demonstrate that CD-BPR not only performs better in predicting exercise outcomes but also provides superior interpretability of estimated student profiles, thus fulfilling both key requirements.
A Methodology Establishing Linear Convergence of Adaptive Gradient Methods under PL Inequality
Authors: Chakrabarti Kushal , Baranwal Mayank
Published: October 2024

Adaptive gradient-descent optimizers are the standard choice for training neural network models. Despite their faster convergence than gradient-descent and remarkable performance in practice, the adaptive optimizers are not as well understood as vanilla gradient-descent. A reason is that the dynamic update of the learning rate that helps in faster convergence of these methods also makes their analysis intricate. Particularly, the simple gradient-descent method converges at a linear rate for a class of optimization problems, whereas the practically faster adaptive gradient methods lack such a theoretical guarantee. The Polyak-Łojasiewicz (PL) inequality is the weakest known class, for which linear convergence of gradient-descent and its momentum variants has been proved. Therefore, in this paper, we prove that AdaGrad and Adam, two well-known adaptive gradient methods, converge linearly when the cost function is smooth and satisfies the PL inequality. Our theoretical framework follows a simple and unified approach, applicable to both batch and stochastic gradients, which can potentially be utilized in analyzing linear convergence of other variants of Adam.
A Federated Large Language Model for Long-Term Time Series Forecasting
Authors: Abdel-Sater Raed , Ben Hamza A.
Published: October 2024

Long-term time series forecasting in centralized environments poses unique challenges regarding data privacy, communication overhead, and scalability. To address these challenges, we propose FedTime, a federated large language model (LLM) tailored for long-range time series prediction. Specifically, we introduce a federated pre-trained LLM with fine-tuning and alignment strategies. Prior to the learning process, we employ K-means clustering to partition edge devices or clients into distinct clusters, thereby facilitating more focused model training. We also incorporate channel independence and patching to better preserve local semantic information, ensuring that important contextual details are retained while minimizing the risk of information loss. We demonstrate the effectiveness of our FedTime model through extensive experiments on various real-world forecasting benchmarks, showcasing substantial improvements over recent approaches. In addition, we demonstrate the efficiency of FedTime in streamlining resource usage, resulting in reduced communication overhead.
Learning Joint Models of Prediction and Optimization
Authors: Kotary James , Di Vito Vincenzo , Christopher Jacob , Van Hentenryck Pascal , Fioretto Ferdinando
Published: October 2024

The Predict-Then-Optimize framework uses machine learning models to predict unknown parameters of an optimization problem from exogenous features before solving. This setting is common to many real-world decision processes, and recently it has been shown that decision quality can be substantially improved by solving and differentiating the optimization problem within an end-to-end training loop. However, this approach requires significant computational effort in addition to handcrafted, problem-specific rules for backpropagation through the optimization step, challenging its applicability to a broad class of optimization problems. This paper proposes an alternative method, in which optimal solutions are learned directly from the observable features by joint predictive models. The approach is generic, and based on an adaptation of the Learning-to-Optimize paradigm, from which a rich variety of existing techniques can be employed. Experimental evaluations show the ability of several Learning-to-Optimize methods to provide efficient and accurate solutions to an array of challenging Predict-Then-Optimize problems.
Be Persistent: Towards a Unified Solution for Mitigating Shortcuts in Deep Learning
Authors: Dolatabadi Hadi M. , Erfani Sarah M. , Leckie Christopher
Published: October 2024

Deep neural networks (DNNs) are vulnerable to shortcut learning: rather than learning the intended task, they tend to draw inconclusive relationships between their inputs and outputs. Shortcut learning is ubiquitous among many failure cases of neural networks, and traces of this phenomenon can be seen in their generalizability issues, domain shift, adversarial vulnerability, and even bias towards majority groups. In this paper, we argue that this commonality in the cause of various DNN issues creates a significant opportunity that should be leveraged to find a unified solution for shortcut learning. To this end, we outline the recent advances in topological data analysis (TDA), and persistent homology (PH) in particular, to sketch a unified roadmap for detecting shortcuts in deep learning. We demonstrate our arguments by investigating the topological features of computational graphs in DNNs using two cases of unlearnable examples and bias in decision-making as our test studies. Our analysis of these two failure cases of DNNs reveals that finding a unified solution for shortcut learning in DNNs is not out of reach, and TDA can play a significant role in forming such a framework.
Large Language Model Agentic Approach to Fact Checking and Fake News Detection
Authors: Li Xinyi , Zhang Yongfeng , Malthouse Edward C.
Published: October 2024

In the current digital era, the rapid spread of misinformation on online platforms presents significant challenges to societal well-being, public trust, and democratic processes, influencing critical decision making and public opinion. To address these challenges, there is a growing need for automated fake news detection mechanisms. Pre-trained large language models (LLMs) have demonstrated exceptional capabilities across various natural language processing (NLP) tasks, prompting exploration into their potential for verifying news claims. Instead of employing LLMs in a non-agentic way, where LLMs generate responses based on direct prompts in a single shot, our work introduces FactAgent, an agentic approach of utilizing LLMs for fake news detection. FactAgent enables LLMs to emulate human expert behavior in verifying news claims without any model training, following a structured workflow. This workflow breaks down the complex task of news veracity checking into multiple sub-steps, where LLMs complete simple tasks using their internal knowledge or external tools. At the final step of the workflow, LLMs integrate all findings throughout the workflow to determine the news claim’s veracity. Compared to manual human verification, FactAgent offers enhanced efficiency. Experimental studies demonstrate the effectiveness of FactAgent in verifying claims without the need for any training process. Moreover, FactAgent provides transparent explanations at each step of the workflow and during final decision-making, offering insights into the reasoning process of fake news detection for end users. FactAgent is highly adaptable, allowing for straightforward updates to its tools that LLMs can leverage within the workflow, as well as updates to the workflow itself using domain knowledge. This adaptability enables FactAgent’s application to news verification across various domains.
Adaptive Fraud Detection on e-Commerce Platforms
Authors: Tang Shaowen , Wong Raymond K.
Published: October 2024

Fraud detection has attracted massive research interests due to its broad applications, especially on e-commerce websites, where fraudulent users interact with products unfairly to mislead potential customers for monetary incentives. Previous studies have made significant research progress on fraud detection, while only a few of them pay attention to adaptability and scalability of their methods. As a result, most of prior fraud detectors cannot perform consistently well across different scenarios due to issues such as sparsity, pattern inconsistency and camouflage. To address these issues, this paper proposes an adaptive algorithm called AdaptFD to effectively detect fraudulent users on e-commerce platforms. In particular, we first tackle sparsity issue with finer granularity by inferring relevant coefficients on entity level during runtime. Next, variable regularization terms are designed for pattern inconsistency tolerance. Lastly, we analyze the behavior patterns of users to reveal camouflage of fraudsters. Together, AdaptFD can adapt to different scenarios and handle with large-scale data efficiently. We also provide theoretical guarantees for our algorithm in linear time complexity and training convergence. Extensive experiments on four real-world datasets demonstrate that AdaptFD outperforms other state-of-the-art baselines. It’s worth noticing that our algorithm achieves up to 560x speedup on the real-world dataset, compared with the most related baseline.
STPDN: Spatio-Temporal Pattern Decomposition Network with Fluctuation Awareness for Robust Traffic Flow Forecasting
Authors: Zhang Xudong , Peng Peng , Chen Xuewen , Wu Yulei , Shen Lingdong , Tang Haina
Published: October 2024

The significance of traffic prediction in modern urban life has become increasingly prominent. Accurate traffic forecasting improves urban traffic management and enhances road resource utilization. In recent years, many models have introduced spatio-temporal contextual embeddings to distinguish between different time steps and spatial nodes. However, these models often overlook anomalous fluctuations in traffic data due to data imbalance. Consequently, performance declines when encountering uncommon situations, especially those caused by unexpected traffic accidents. To maintain overall performance while being aware of anomalous fluctuations, we propose STPDN, a dual-branch Spatio-Temporal Pattern Decomposition Graph Neural Network. Specifically, We introduce latent variables to characterize the distribution of latent patterns in traffic sequences, enabling the model to distinguish regular patterns and anomalous fluctuations without supervised information specifically targeting anomalies. Subsequently, we develop a resilient graph generator capable of producing dynamic spatio-temporal graphs, facilitating the propagation of impacts caused by dynamic fluctuations. Finally, we achieve more comprehensive and robust predictions by fusing regular patterns and anomalous fluctuations. Evaluation of real-world and simulated datasets shows that our model outperforms others, offering more reliable prediction solutions for urban traffic management systems, particularly in handling unforeseen traffic events. The code can be found at https://github.com/dhxdla/PyTorch-implementation-of-the-STPDN.git.
Estimating the Robustness Radius for Randomized Smoothing with 100× Sample Efficiency
Authors: Seferis Emmanouil , Kollias Stefanos , Cheng Chih-Hong
Published: October 2024

Randomized smoothing (RS) has successfully been used to improve the robustness of predictions for deep neural networks (DNNs) by adding random noise to create multiple variations of an input, followed by deciding the consensus. To understand if an RS-enabled DNN is effective in the sampled input domains, it is mandatory to sample data points within the operational design domain, acquire the point-wise certificate regarding robustness radius, and compare it with pre-defined acceptance criteria. Consequently, ensuring that a point-wise robustness certificate for any given data point is obtained relatively cost-effectively is crucial. This work demonstrates that reducing the number of samples by one or two orders of magnitude can still enable the computation of a slightly smaller robustness radius (commonly ≈20% radius reduction) with the same confidence. We provide the mathematical foundation for explaining the phenomenon while experimentally showing promising results on the standard CIFAR-10 and ImageNet datasets.
Model Priming with Triplet Loss for Few-Shot Emotion Classification in Text
Authors: Lemmens Jens , Daelemans Walter
Published: October 2024

Automatically detecting emotions in text data is a challenging task, especially when only little supervised training data is available. Therefore, we attempt to boost model performance in few-shot experiments by learning emotion label information. We explore triplet loss for emotion detection from text, and utilize this technique to cluster text representations that express the same emotion in the embedding space before learning the classification task. We show that this method, which we call emotion priming, outperforms baseline results, multi-task fine-tuning with cross-entropy loss, and an existing label infusion method that adds label words to the input sequence to alter the model’s attention weights. An analysis of the emotion class representations after priming shows that the observed performance gain can be attributed to the redistribution of the text representations. The results also indicate that this method is robust in datasets that contain many classes and few examples per class. In contrast to earlier work, we found that the label infusion method leads to a substantial performance decrease compared to the baseline model, especially for the datasets with more complex label schemes. Finally, we report results for zero-shot experiments with ChatGPT as a larger alternative to smaller fine-tuned language models, and show that it fails to produce accurate results, indicating the complexity of the studied task.
Reduce, Reuse, Recycle: Categories for Compositional Reinforcement Learning
Authors: Bakirtzis Georgios , Savvas Michail , Zhao Ruihan , Chinchali Sandeep , Topcu Ufuk
Published: October 2024

In reinforcement learning, conducting task composition by forming cohesive, executable sequences from multiple tasks remains challenging. However, the ability to (de)compose tasks is a linchpin in developing robotic systems capable of learning complex behaviors. Yet, compositional reinforcement learning is beset with difficulties, including the high dimensionality of the problem space, scarcity of rewards, and absence of system robustness after task composition. To surmount these challenges, we view task composition through the prism of category theory—a mathematical discipline exploring structures and their compositional relationships. The categorical properties of Markov decision processes untangle complex tasks into manageable sub-tasks, allowing for strategical reduction of dimensionality, facilitating more tractable reward structures, and bolstering system robustness. Experimental results support the categorical theory of reinforcement learning by enabling skill reduction, reuse, and recycling when learning complex robotic arm tasks.
Efficient Initial Data Selection and Labeling for Multi-Class Classification Using Topological Analysis
Authors: Hadjadj Lies , Devijver Emilie , Molinier Rémi , Amini Massih-Reza
Published: October 2024

Machine learning techniques often require large labeled training sets to attain optimal performance. However, acquiring labeled data can pose challenges in practical scenarios. Pool-based active learning methods aim to select the most relevant data points for training from a pool of unlabeled data. Nonetheless, these methods heavily rely on the initial labeled dataset, often chosen randomly. In our study, we introduce a novel approach specifically tailored for multi-class classification tasks, utilizing Proper Topological Regions (PTR) derived from topological data analysis (TDA) to efficiently identify the initial set of points for labeling. Through experiments on various benchmark datasets, we demonstrate the efficacy of our method and its competitive performance compared to traditional approaches, as measured by average balanced classification accuracy.
Synthesis of Reward Machines for Multi-Agent Equilibrium Design
Authors: Najib Muhammad , Perelli Giuseppe
Published: October 2024

Mechanism design is a well-established game-theoretic paradigm for designing games to achieve desired outcomes. This paper addresses a closely related but distinct concept, equilibrium design. Unlike mechanism design, the designer’s authority in equilibrium design is more constrained; she can only modify the incentive structures in a given game to achieve certain outcomes without the ability to create the game from scratch. We study the problem of equilibrium design using dynamic incentive structures, known as reward machines. We use weighted concurrent game structures for the game model, with goals (for the players and the designer) defined as mean-payoff objectives. We show how reward machines can be used to represent dynamic incentives that allocate rewards in a manner that optimises the designer’s goal. We also introduce the main decision problem within our framework, the payoff improvement problem. This problem essentially asks whether there exists a dynamic incentive (represented by some reward machine) that can improve the designer’s payoff by more than a given threshold value. We present two variants of the problem: strong and weak. We demonstrate that both can be solved in polynomial time using a Turing machine equipped with an NP oracle. Furthermore, we also establish that these variants are either NP-hard or coNP-hard. Finally, we show how to synthesise the corresponding reward machine if it exists.
An Online Incremental Learning Approach for Configuring Multi-arm Bandits Algorithms
Authors: Alsomali Mohammad Essa , Rodrigues-Filho Roberto , Marcolino Leandro Soriano , Porter Barry
Published: October 2024

This paper introduces Dynamic Bayesian Optimisation for Multi-Arm Bandits (DBO-MAB), an algorithm that dynamically adapts hyperparameters of multi-arm bandit algorithms using incremental Bayesian optimisation. DBO-MAB addresses the challenge of tuning hyperparameters in uncertain and dynamic environments, particularly for applications like web server optimisation. It uses a dynamic range adjustment approach based on the interquartile mean (IQM) of observed rewards to focus the search space on promising regions. Evaluated across diverse static and dynamic environments, DBO-MAB outperforms state-of-the-art algorithms such as Bootstrapped UCB and f-Discounted-Sliding-Window Thompson Sampling, reducing average response time by ≈55%.
Bridging Continual Learning of Motion and Self-Supervised Representations
Authors: Tiezzi Matteo , Marullo Simone , Betti Alessandro , Casoni Michele , Melacci Stefano
Published: October 2024

Efficiently learning unsupervised pixel-wise visual representations is crucial for training agents that can perceive their environment without relying on heavy human supervision or abundant annotated data. Motivated by recent work that promotes motion as a key source of information in representation learning, we propose a novel instance of contrastive criterions over time and space. In our architecture, pixel-wise motion field and representations are extracted by neural models, trained from scratch in an integrated fashion. Learning proceeds online over time, exploiting also a momentum-based moving average to update the feature extractor, without replaying any large buffers of past data. Experiments on real-world videos and on a recently introduced benchmark, with photorealistic streams generated from a 3D environment, confirm that the proposed model can learn to estimate motion and jointly develop representations. Our model nicely encodes the variable appearance of the visual information in space and time, significantly overcoming a recent approach and it also compares favourably with convolutional and Transformer-based networks, offline-pre-trained on large collections of supervised and unsupervised images.
DINEX: Interpretable NLP via Diversified Natural Language Explanations
Authors: Zhou Xugang , Su JinDian , Gu WeiZheng
Published: October 2024

Natural Language Explanations (NLE) are becoming increasingly important in Interpretable Natural Language Processing, which can clarify the reasoning process and improve performance. It is essential to utilize human-authored gold explanations to improve the quality of explanations produced by the generator. However, the gold standard explanations within the same dataset are produced by a fixed group of annotators, leading to a more homogeneous perspective and style. This has inspired us to improve the quality of generated explanations by enhancing the diversity of the training set. Based on this, we introduce DINEX, a two-stage framework comprising a diversified explanation generator and an explanation-aware predictor, suitable for any task related to NLE. The first stage of DINEX augments the generator’s training set through two approaches: Semantic Similarity Sampling (SSS) and Structural Variety Generation (SVG). This enables the generator to learn how to produce NLE from diverse perspectives and styles. In the second stage, DINEX improves the predictor’s ability to capture the complementary semantics between contexts and explanations. It also reduces the impact of noise on predictions through dynamic perturbations. We conduct experiments on four datasets in the domains of Question Answering and Reasoning. The results show that DINEX achieves an average performance improvement of 5.49%, establishing a new state-of-the-art on the ComVE dataset. Evaluations by human evaluators and Large Language Models (LLMs) demonstrate that DINEX-generated explanations surpass the baseline in quality across 62.4% of the test samples.
Development of an Interpretable Uni-Null Neuron-Based Evolving Fuzzy Neural Network for Age Group Identification in Respondents with Diabetes
Authors: de Campos Souza Paulo Vitor , Dragoni Mauro
Published: October 2024

In the domain of healthcare and well-being, the fusion of machine learning and data collection through medical examinations has propelled significant advancements in diabetes monitoring. Diabetes, a prevalent and intricate health condition, has garnered increasing attention due to its substantial impact on individuals’ mental and physical well-being. The use of medical examinations for real-time diabetes assessment has become pivotal, with various physiological monitoring capabilities aiding in this endeavor. Machine learning, as a subset of artificial intelligence, has further elevated the precision and effectiveness of diabetes monitoring by extracting meaningful insights from the extensive and intricate data collected through these examinations. This paper introduces a novel and interpretable computational model known as the Evolving Fuzzy Neural Network Uni-Nullneuron-Based Approach (EFNN-UniNull). Comprising three interconnected layers, this model collaboratively produces classification outcomes while concurrently providing insightful interpretations of relationships within the age group identification of diabetes patients dataset. The fuzzification method based on grid partition helps in obtaining adequate knowledge about the problem. The model underwent a comparative analysis against evolving neuro-fuzzy systems, demonstrating results approaching 85% accuracy. Notably, the model extracted knowledge based on fuzzy rules pertinent to diabetes identification.
SaGess: A Sampling Graph Denoising Diffusion Model for Scalable Graph Generation
Authors: Limnios Stratis , Selvaraj Praveen , Cucuringu Mihai , Maple Carsten , Reinert Gesine , Elliott Andrew
Published: October 2024

Denoising diffusion generative models are state-of-the-art methods for generating synthetic images that have also proved successful in tabular and graph synthetic data generation. However, their computational complexity has limited the application of these techniques to graph data, focusing usually on smaller graphs, such as those used in molecular modeling. In this paper, we propose SaGess, a discrete denoising diffusion approach, which is able to generate large real-world networks. Through a generalized divide-and-conquer framework, SaGess overcomes the scaling limitations of the diffusion model DiGress, by sampling a covering of subgraphs of the initial graph, training a DiGress module, and finally reconstructing a synthetic graph using the subgraphs that have been generated using the DiGress module. We evaluate the quality of the synthetic data sets against several competitor methods by comparing graph statistics between the original and synthetic samples, as well as evaluating the utility of the synthetic data set produced by using it to train a task-driven model, namely link prediction. In our experiments, SaGess  outperforms most of the one-shot state-of-the-art graph generating methods by a significant factor, both on the graph metrics and on the link prediction task.
Thresholded Lexicographic Ordered Multiobjective Reinforcement Learning
Authors: Tercan Alperen , Prabhu Vinayak S.
Published: October 2024

Lexicographic multi-objective problems, which impose a lexicographic importance order over the objectives, arise in many real-life scenarios. Existing Reinforcement Learning work directly addressing lexicographic tasks has been scarce. The few proposed approaches were all noted to be heuristics without theoretical guarantees as the Bellman equation is not applicable to them. Additionally, the practical applicability of these prior approaches also suffers from various issues such as not being able to reach the goal state. While some of these issues have been known before, in this work we investigate further shortcomings and propose fixes for improving practical performance in many cases. We also present a policy optimization approach using our Lexicographic Projection Algorithm (LPA) that has the potential to address these theoretical and practical concerns. Finally, we demonstrate our proposed algorithms on benchmark problems.
Subsystem Discovery in High-Dimensional Time-Series Using Masked Autoencoders
Authors: Sarapisto Teemu , Wei Haoyu , Heljanko Keijo , Klami Arto , Ruotsalainen Laura
Published: October 2024

Deep neural networks are increasingly used for time series tasks, yet they often struggle to interpretably model high-dimensional data. In this context, we consider the task of learning easy to understand connections between time-series variables, and organizing them into subsystems, directly from observed data. Our approach reconstructs multivariate time-series with a masked autoencoder, where all information between individual variables is mediated by a learned adjacency matrix. This intuitive pairwise relationship enables grouping of variables without prior knowledge of cluster quantity or size, and is particularly useful for analyzing complex sensor systems with unknown structural interdependencies. Our method simultaneously learns a useful signal representation and aids in understanding the underlying processes. We show that we can learn the correct subsystems from simulated data, and demonstrate identification of plausible subsystem structure from high-dimensional real-world data. In addition, we show that the model retains high predictive performance.
IFH: A Diffusion Framework for Flexible Design of Graph Generative Models
Authors: Cognolato Samuel , Sperduti Alessandro , Serafini Luciano
Published: October 2024

Graph generative models can be classified into two prominent families: one-shot models, which generate a graph in one go, and sequential models, which generate a graph by successive additions of nodes and edges. Ideally, between these two extreme models lies a continuous range of models that adopt different levels of sequentiality. This paper proposes a graph generative model, called Insert-Fill-Halt (IFH), that supports the specification of a sequentiality degree. IFH is based upon the theory of Denoising Diffusion Probabilistic Models (DDPM), designing a node removal process that gradually destroys a graph. An insertion process learns to reverse this removal process by inserting arcs and nodes according to the specified sequentiality degree. We evaluate the performance of IFH in terms of quality, run time, and memory, depending on different sequentiality degrees. We also show that using DiGress, a diffusion-based one-shot model, as a generative step in IFH leads to improvement to the model itself, and is competitive with the current state-of-the-art.
B2MAPO: A Batch-by-Batch Multi-Agent Policy Optimization to Balance Performance and Efficiency
Authors: Zhang Wenjing , Zhang Wei , Hu Wenqing , Wang Yifan
Published: October 2024

Most multi-agent reinforcement learning approaches adopt two types of policy optimization methods that either update policy simultaneously or sequentially. Simultaneously updating policies of all agents introduces non-stationarity problem. Although sequentially updating policies agent-by-agent in an appropriate order improves policy performance, it is prone to low efficiency due to sequential execution, resulting in longer model training and execution time. Intuitively, partitioning policies of all agents according to their interdependence and updating joint policy batch-by-batch can effectively balance performance and efficiency. However, how to determine the optimal batch partition of policies and batch updating order are challenging problems. Firstly, a sequential batched policy updating scheme, B2MAPO (Batch by Batch Multi-Agent Policy Optimization), is proposed with a theoretical guarantee of the monotonic incrementally tightened bound. Secondly, a universal modulized plug-and-play B2MAPO hierarchical framework, which satisfies CTDE principle, is designed to conveniently integrate any MARL models to fully exploit and merge their merits, including policy optimality and inference efficiency. Next, a DAG-based B2MAPO algorithm is devised, which is a carefully designed implementation of B2MAPO framework. The upper layer employs PPO algorithm with attention mechanism to reveal interdependence between policies, and generates DAGs of agents which are used to produce optimal batch sequence through topological sorting. The lower layer trains two joint policies in parallel and minimizes KL divergence between them periodically. One joint policy is sequentially updated according to B2MAPO scheme with batch sequence, another derived joint policy is simultaneously updated by MAPPO. While decentralized execution, only the derived joint policy is adopted for decision-making. Comprehensive experimental results conducted on StarCraftII Multi-agent Challenge and Google Football Research demonstrate the performance of DAG-based B2MAPO algorithm outperforms baseline methods. Meanwhile, compared with A2PO, our algorithm reduces the model training and execution time by 60.4% and 78.7%, respectively.
Artificial Agents Facilitate Human Cooperation Through Indirect Reciprocity
Authors: Pires Alexandre S. , Santos Fernando P.
Published: October 2024

Indirect reciprocity (IR) is a key mechanism to explain cooperation in human populations. With IR, individuals acquire reputations which can be used by others when deciding to cooperate or defect: the costs of cooperation can therefore be outweighed by the long-term benefits of keeping a good reputation. Although IR has been studied assuming populations fully composed of humans, social interactions nowadays involve the ever-increasing presence of artificial agents (AAs) such as social bots, conversational agents or even collaborative robots. It remains unclear how IR dynamics will be affected once artificial agents co-exist with humans. Here we develop a theoretical model to investigate the potential effect of AAs, deployed with a fixed strategy, in the evolving cooperation levels observed in a population. We study settings where AAs are subject to the same reputation update rules as the remaining adaptive agents and settings where AAs have a fixed reputation. We show that introducing a small fraction of AAs with a discriminating strategy (i.e., cooperate only with good agents) increases the cooperation rate in the whole population. Moreover, the positive effect of AAs is exacerbated when these are unconditionally assessed as good. We also demonstrate the vulnerability of cooperation towards purely defecting AAs, and the inefficacy of non-discriminating cooperators in promoting cooperation. Our theoretical work contributes to identify the settings where artificial agents, even with simple hard-coded strategies, can help humans to solve a social dilemma of cooperation.
Multiwinner Temporal Voting with Aversion to Change
Authors: Zech Valentin , Boehmer Niclas , Elkind Edith , Teh Nicholas
Published: October 2024

We study two-stage committee elections where voters have dynamic preferences over candidates; at each stage, a committee is chosen under a given voting rule. We are interested in identifying a winning committee for the second stage that overlaps as much as possible with the first-stage committee. We show a full complexity dichotomy for the class of Thiele rules: this problem is tractable for Approval Voting (AV) and hard for all other Thiele rules (including, in particular, Proportional Approval Voting and the Chamberlin–Courant rule). We extend this dichotomy to the greedy variants of Thiele rules. We also explore this problem from a parameterized complexity perspective for several natural parameters. We complement the theory with experimental analysis: e.g., we investigate the average number of changes in the committee as a function of changes in voters’ preferences and the role of ties.
Meta-Mechanisms for Combinatorial Auctions over Social Networks
Authors: Fang Yuan , Zhang Mengxiao , Liu Jiamou , Khoussainov Bakh
Published: October 2024

Recently there has been a large amount of research designing mechanisms for auction scenarios where the bidders are connected in a social network. Different from the existing studies in this field that focus on specific auction scenarios e.g. single-unit auction and multi-unit auction, this paper considers the following question: is it possible to design a scheme that, given a classical auction scenario and a mechanism M˜ suited for it, produces a mechanism in the network setting that preserves the key properties of M˜? To answer this question, we design meta-mechanisms that provide a uniform way of transforming mechanisms from classical models to mechanisms over networks and prove that the desirable properties are preserved by our meta-mechanisms. Our meta-mechanisms provide solutions to combinatorial auction scenarios in the network setting: (1) combinatorial auction with single-minded buyers and (2) combinatorial auction with general monotone valuation. To the best of our knowledge, this is the first work that designs combinatorial auctions over a social network.
Balancing Efficiency with Equality: Auction Design with Group Fairness Concerns
Authors: Jia Fengjuan , Zhang Mengxiao , Liu Jiamou , Khoussainov Bakh
Published: October 2024

The issue of fairness in AI arises from discriminatory practices in applications like job recommendations and risk assessments, emphasising the need for algorithms that do not discriminate based on group characteristics. This concern is also pertinent to auctions, commonly used for resource allocation, which necessitate fairness considerations. Our study examines auctions with groups distinguished by specific attributes, seeking to (1) define a fairness notion that ensures equitable treatment for all, (2) identify mechanisms that adhere to this fairness while preserving incentive compatibility, and (3) explore the balance between fairness and seller’s revenue. We introduce two fairness notions—group fairness and individual fairness—and propose two corresponding auction mechanisms: the Group Probability Mechanism, which meets group fairness and incentive criteria, and the Group Score Mechanism, which also encompasses individual fairness. Through experiments, we validate these mechanisms’ effectiveness in promoting fairness and examine their implications for seller revenue.