pwc-archive/papers-with-abstracts huggingface dataset:
Soft regression trees: a model variant and a decomposition training algorithm
Authors: Antonio Consolo, Edoardo Amaldi, Andrea Manno
Date: 2025-01-10 00:00:00
Tasks: regression

Decision trees are widely used for classification and regression tasks in a variety of application fields due to their interpretability and good accuracy. During the past decade, growing attention has been devoted to globally optimized decision trees with deterministic or soft splitting rules at branch nodes, which are trained by optimizing the error function over all the tree parameters. In this work, we propose a new variant of soft multivariate regression trees (SRTs) where, for every input vector, the prediction is defined as the linear regression associated to a single leaf node, namely, the leaf node obtained by routing the input vector from the root along the branches with higher probability. SRTs exhibit the conditional computational property, i.e., each prediction depends on a small number of nodes (parameters), and our nonlinear optimization formulation for training them is amenable to decomposition. After showing a universal approximation result for SRTs, we present a decomposition training algorithm including a clustering-based initialization procedure and a heuristic for reassigning the input vectors along the tree. Under mild assumptions, we establish asymptotic convergence guarantees. Experiments on 15 wellknown datasets indicate that our SRTs and decomposition algorithm yield higher accuracy and robustness compared with traditional soft regression trees trained using the nonlinear optimization formulation of Blanquero et al., and a significant reduction in training times as well as a slightly better average accuracy compared with the mixed-integer optimization approach of Bertsimas and Dunn. We also report a comparison with the Random Forest ensemble method.
Encoded Spatial Attribute in Multi-Tier Federated Learning
Authors: Asfia Kawnine, Francis Palma, Seyed Alireza Rahimi Azghadi, Hung Cao
Date: 2025-01-10 00:00:00
Tasks: Attribute, Federated Learning

This research presents an Encoded Spatial Multi-Tier Federated Learning approach for a comprehensive evaluation of aggregated models for geospatial data. In the client tier, encoding spatial information is introduced to better predict the target outcome. The research aims to assess the performance of these models across diverse datasets and spatial attributes, highlighting variations in predictive accuracy. Using evaluation metrics such as accuracy, our research reveals insights into the complexities of spatial granularity and the challenges of capturing underlying patterns in the data. We extended the scope of federated learning (FL) by having multi-tier along with the functionality of encoding spatial attributes. Our N-tier FL approach used encoded spatial data to aggregate in different tiers. We obtained multiple models that predicted the different granularities of spatial data. Our findings underscore the need for further research to improve predictive accuracy and model generalization, with potential avenues including incorporating additional features, refining model architectures, and exploring alternative modeling approaches. Our experiments have several tiers representing different levels of spatial aspects. We obtained accuracy of 75.62% and 89.52% for the global model without having to train the model using the data constituted with the designated tier. The research also highlights the importance of the proposed approach in real-time applications.
Q-MAML: Quantum Model-Agnostic Meta-Learning for Variational Quantum Algorithms
Authors: Junyong Lee, Jeihee Cho, Shiho Kim
Date: 2025-01-10 00:00:00
Tasks: Meta-Learning

In the Noisy Intermediate-Scale Quantum (NISQ) era, using variational quantum algorithms (VQAs) to solve optimization problems has become a key application. However, these algorithms face significant challenges, such as choosing an effective initial set of parameters and the limited quantum processing time that restricts the number of optimization iterations. In this study, we introduce a new framework for optimizing parameterized quantum circuits (PQCs) that employs a classical optimizer, inspired by Model-Agnostic Meta-Learning (MAML) technique. This approach aim to achieve better parameter initialization that ensures fast convergence. Our framework features a classical neural network, called Learner}, which interacts with a PQC using the output of Learner as an initial parameter. During the pre-training phase, Learner is trained with a meta-objective based on the quantum circuit cost function. In the adaptation phase, the framework requires only a few PQC updates to converge to a more accurate value, while the learner remains unchanged. This method is highly adaptable and is effectively extended to various Hamiltonian optimization problems. We validate our approach through experiments, including distribution function mapping and optimization of the Heisenberg XYZ Hamiltonian. The result implies that the Learner successfully estimates initial parameters that generalize across the problem space, enabling fast adaptation.
Discovery of sustainable energy materials via the machine-learned material space
Authors: Malte Grunert, Max Großmann, Erich Runge
Date: 2025-01-10 00:00:00
Tasks: Dimensionality Reduction, Graph Attention

Does a machine learning model actually gain an understanding of the material space? We answer this question in the affirmative on the example of the OptiMate model, a graph attention network trained to predict the optical properties of semiconductors and insulators. By applying the UMAP dimensionality reduction technique to its latent embeddings, we demonstrate that the model captures a nuanced and interpretable representation of the materials space, reflecting chemical and physical principles, without any user-induced bias. This enables clustering of almost 10,000 materials based on optical properties and chemical similarities. Beyond this understanding, we demonstrate how the learned material space can be used to identify more sustainable alternatives to critical materials in energy-related technologies, such as photovoltaics. These findings demonstrate the dual utility of machine learning models in materials science: Accurately predicting material properties while providing insights into the underlying materials space. The approach demonstrates the broader potential of leveraging learned materials spaces for the discovery and design of materials for diverse applications, and is easily applicable to any state-of-the-art machine learning model.
Text2Playlist: Generating Personalized Playlists from Text on Deezer
Authors: Mathieu Delcluze, Antoine Khoury, Clémence Vast, Valerio Arnaudo, Léa Briand, Walid Bendada, Thomas Bouabça
Date: 2025-01-10 00:00:00
Tasks: Information Retrieval, Music Information Retrieval, Navigate, Recommendation Systems, Retrieval

The streaming service Deezer heavily relies on the search to help users navigate through its extensive music catalog. Nonetheless, it is primarily designed to find specific items and does not lead directly to a smooth listening experience. We present Text2Playlist, a stand-alone tool that addresses these limitations. Text2Playlist leverages generative AI, music information retrieval and recommendation systems to generate query-specific and personalized playlists, successfully deployed at scale.
Collaborative Content Moderation in the Fediverse
Authors: Haris Bin Zia, Aravindh Raman, Ignacio Castro, Gareth Tyson
Date: 2025-01-10 00:00:00
Tasks: Federated Learning

The Fediverse, a group of interconnected servers providing a variety of interoperable services (e.g. micro-blogging in Mastodon) has gained rapid popularity. This sudden growth, partly driven by Elon Musk's acquisition of Twitter, has created challenges for administrators though. This paper focuses on one particular challenge: content moderation, e.g. the need to remove spam or hate speech. While centralized platforms like Facebook and Twitter rely on automated tools for moderation, their dependence on massive labeled datasets and specialized infrastructure renders them impractical for decentralized, low-resource settings like the Fediverse. In this work, we design and evaluate FedMod, a collaborative content moderation system based on federated learning. Our system enables servers to exchange parameters of partially trained local content moderation models with similar servers, creating a federated model shared among collaborating servers. FedMod demonstrates robust performance on three different content moderation tasks: harmful content detection, bot content detection, and content warning assignment, achieving average per-server macro-F1 scores of 0.71, 0.73, and 0.58, respectively.
A Neighbor-based Approach to Pitch Ownership Models in Soccer
Authors: Tiago Mendes-Neves, Luís Meireles, João Mendes-Moreira
Date: 2025-01-10 00:00:00
Tasks: Pitch control

Pitch ownership models allow many types of analysis in soccer and provide valuable assistance to tactical analysts in understanding the game's dynamics. The novelty they provide over event-based analysis is that tracking data incorporates context that event-based data does not possess, like player positioning. This paper proposes a novel approach to building pitch ownership models in soccer games using the K-Nearest Neighbors (KNN) algorithm. Our approach provides a fast inference mechanism that can model different approaches to pitch control using the same algorithm. Despite its flexibility, it uses only three hyperparameters to tune the model, facilitating the tuning process for different player skill levels. The flexibility of the approach allows for the emulation of different methods available in the literature by adjusting a small number of parameters, including adjusting for different levels of uncertainty. In summary, the proposed model provides a new and more flexible strategy for building pitch ownership models, extending beyond just replicating existing algorithms, and can provide valuable insights for tactical analysts and open up new avenues for future research. We thoroughly visualize several examples demonstrating the presented models' strengths and weaknesses. The code is available at github.com/nvsclub/KNNPitchControl.
Neural Network Verification is a Programming Language Challenge
Authors: Lucas C. Cordeiro, Matthew L. Daggitt, Julien Girard-Satabin, Omri Isac, Taylor T. Johnson, Guy Katz, Ekaterina Komendantskaya, Augustin Lemesle, Edoardo Manino, Artjoms Šinkarovs, Haoze Wu
Date: 2025-01-10 00:00:00

Neural network verification is a new and rapidly developing field of research. So far, the main priority has been establishing efficient verification algorithms and tools, while proper support from the programming language perspective has been considered secondary or unimportant. Yet, there is mounting evidence that insights from the programming language community may make a difference in the future development of this domain. In this paper, we formulate neural network verification challenges as programming language challenges and suggest possible future solutions.
"Cause" is Mechanistic Narrative within Scientific Domains: An Ordinary Language Philosophical Critique of "Causal Machine Learning"
Authors: Vyacheslav Kungurtsev, Leonardo Christov Moore, Gustav Sir, Martin Krutsky
Date: 2025-01-10 00:00:00

Causal Learning has emerged as a major theme of research in statistics and machine learning in recent years, promising specific computational techniques to apply to datasets that reveal the true nature of cause and effect in a number of important domains. In this paper we consider the epistemology of recognizing true cause and effect phenomena. We apply the Ordinary Language method of engaging on the customary use of the word 'cause' to investigate valid semantics of reasoning about cause and effect. We recognize that the grammars of cause and effect are fundamentally distinct in form across scientific domains, yet they maintain a consistent and central function. This function can best be described as the mechanism underlying fundamental forces of influence as considered prominent in the respective scientific domain. We demarcate 1) physics and engineering as domains wherein mathematical models are sufficient to comprehensively describe causality, 2) biology as introducing challenges of emergence while providing opportunities for showing consistent mechanisms across scale, and 3) the social sciences as introducing grander difficulties for establishing models of low prediction error but providing, through Hermeneutics, the potential for findings that are still instrumentally useful to individuals. We posit that definitive causal claims regarding a given phenomenon (writ large) can only come through an agglomeration of consistent evidence across multiple domains. This presents important methodological questions as far as harmonizing between language games and emergence across scales. Given the role of epistemic hubris in the contemporary crisis of credibility in the sciences, exercising greater caution as far as communicating precision as to the real degree of certainty certain evidence provides for rich collections of open problems in optimizing integration of different findings.
Orthogonal projection-based regularization for efficient model augmentation
Authors: Bendegúz M. Györök, Jan H. Hoekstra, Johan Kon, Tamás Péni, Maarten Schoukens, Roland Tóth
Date: 2025-01-10 00:00:00
Tasks: model

Deep-learning-based nonlinear system identification has shown the ability to produce reliable and highly accurate models in practice. However, these black-box models lack physical interpretability, and a considerable part of the learning effort is often spent on capturing already expected/known behavior of the system, that can be accurately described by first-principles laws of physics. A potential solution is to directly integrate such prior physical knowledge into the model structure, combining the strengths of physics-based modeling and deep-learning-based identification. The most common approach is to use an additive model augmentation structure, where the physics-based and the machine-learning (ML) components are connected in parallel, i.e., additively. However, such models are overparametrized, training them is challenging, potentially causing the physics-based part to lose interpretability. To overcome this challenge, this paper proposes an orthogonal projection-based regularization technique to enhance parameter learning and even model accuracy in learning-based augmentation of nonlinear baseline models.
Fine-tuning is Not Fine: Mitigating Backdoor Attacks in GNNs with Limited Clean Data
Authors: Jiale Zhang, Bosen Rao, Chengcheng Zhu, Xiaobing Sun, Qingming Li, Haibo Hu, Xiapu Luo, Qingqing Ye, Shouling Ji
Date: 2025-01-10 00:00:00
Tasks: Graph Attention

Graph Neural Networks (GNNs) have achieved remarkable performance through their message-passing mechanism. However, recent studies have highlighted the vulnerability of GNNs to backdoor attacks, which can lead the model to misclassify graphs with attached triggers as the target class. The effectiveness of recent promising defense techniques, such as fine-tuning or distillation, is heavily contingent on having comprehensive knowledge of the sufficient training dataset. Empirical studies have shown that fine-tuning methods require a clean dataset of 20% to reduce attack accuracy to below 25%, while distillation methods require a clean dataset of 15%. However, obtaining such a large amount of clean data is commonly impractical. In this paper, we propose a practical backdoor mitigation framework, denoted as GRAPHNAD, which can capture high-quality intermediate-layer representations in GNNs to enhance the distillation process with limited clean data. To achieve this, we address the following key questions: How to identify the appropriate attention representations in graphs for distillation? How to enhance distillation with limited data? By adopting the graph attention transfer method, GRAPHNAD can effectively align the intermediate-layer attention representations of the backdoored model with that of the teacher model, forcing the backdoor neurons to transform into benign ones. Besides, we extract the relation maps from intermediate-layer transformation and enforce the relation maps of the backdoored model to be consistent with that of the teacher model, thereby ensuring model accuracy while further reducing the influence of backdoors. Extensive experimental results show that by fine-tuning a teacher model with only 3% of the clean data, GRAPHNAD can reduce the attack success rate to below 5%.
AdaPRL: Adaptive Pairwise Regression Learning with Uncertainty Estimation for Universal Regression Tasks
Authors: Fuhang Liang, Rucong Xu, Deng Lin
Date: 2025-01-10 00:00:00
Tasks: Age Estimation, Multi-Task Learning, Multivariate Time Series Forecasting, Natural Language Understanding, Recommendation Systems, regression, Time Series, Time Series Forecasting

Current deep regression models usually learn in point-wise way that treat each sample as an independent input, neglecting the relative ordering among different data. Consequently, the regression model could neglect the data 's interrelationships, potentially resulting in suboptimal performance. Moreover, the existence of aleatoric uncertainty in the training data may drive the model to capture non-generalizable patterns, contributing to increased overfitting. To address these issues, we propose a novel adaptive pairwise learning framework (AdaPRL) for regression tasks which leverages the relative differences between data points and integrates with deep probabilistic models to quantify the uncertainty associated with the predictions. Additionally, we adapt AdaPRL for applications in multi-task learning and multivariate time series forecasting. Extensive experiments with several real-world regression datasets including recommendation systems, age estimation, time series forecasting, natural language understanding, finance, and industry datasets show that AdaPRL is compatible with different backbone networks in various tasks and achieves state-of-the-art performance on the vast majority of tasks, highlighting its notable potential including enhancing prediction accuracy and ranking ability, increasing generalization capability, improving robustness to noisy data, improving resilience to reduced data, and enhancing interpretability, etc.
STHFL: Spatio-Temporal Heterogeneous Federated Learning
Authors: Shunxin Guo, Hongsong Wang, Shuxia Lin, Xu Yang, Xin Geng
Date: 2025-01-10 00:00:00
Tasks: Federated Learning

Federated learning is a new framework that protects data privacy and allows multiple devices to cooperate in training machine learning models. Previous studies have proposed multiple approaches to eliminate the challenges posed by non-iid data and inter-domain heterogeneity issues. However, they ignore the \textbf{spatio-temporal} heterogeneity formed by different data distributions of increasing task data in the intra-domain. Moreover, the global data is generally a long-tailed distribution rather than assuming the global data is balanced in practical applications. To tackle the \textbf{spatio-temporal} dilemma, we propose a novel setting named \textbf{Spatio-Temporal Heterogeneity} Federated Learning (STHFL). Specially, the Global-Local Dynamic Prototype (GLDP) framework is designed for STHFL. In GLDP, the model in each client contains personalized layers which can dynamically adapt to different data distributions. For long-tailed data distribution, global prototypes are served as complementary knowledge for the training on classes with few samples in clients without leaking privacy. As tasks increase in clients, the knowledge of local prototypes generated in previous tasks guides for training in the current task to solve catastrophic forgetting. Meanwhile, the global-local prototypes are updated through the moving average method after training local prototypes in clients. Finally, we evaluate the effectiveness of GLDP, which achieves remarkable results compared to state-of-the-art methods in STHFL scenarios.
rmlnomogram: An R package to construct an explainable nomogram for any machine learning algorithms
Authors: Herdiantri Sufriyana, Emily Chia-Yu Su
Date: 2025-01-10 00:00:00

Background: Current nomogram can only be created for regression algorithm. Providing nomogram for any machine learning (ML) algorithms may accelerate model deployment in clinical settings or improve model availability. We developed an R package and web application to construct nomogram with model explainability of any ML algorithms. Methods: We formulated a function to transform an ML prediction model into a nomogram, requiring datasets with: (1) all possible combinations of predictor values; (2) the corresponding outputs of the model; and (3) the corresponding explainability values for each predictor (optional). Web application was also created. Results: Our R package could create 5 types of nomograms for categorical predictors and binary outcome without probability (1), categorical predictors and binary outcome with probability (2) or continuous outcome (3), and categorical with single numerical predictors and binary outcome with probability (4) or continuous outcome (5). Respectively, the first and remaining types optimally allowed maximum 15 and 5 predictors with maximum 3,200 combinations. Web application is provided with such limits. The explainability values were possible for types 2 to 5. Conclusions: Our R package and web application could construct nomogram with model explainability of any ML algorithms using a fair number of predictors.
Development and Comparison of Model-Based and Data-Driven Approaches for the Prediction of the Mechanical Properties of Lattice Structures
Authors: Chiara Pasini, Oscar Ramponi, Stefano Pandini, Luciana Sartore, Giulia Scalet
Date: 2025-01-10 00:00:00

Lattice structures have great potential for several application fields ranging from medical and tissue engineering to aeronautical one. Their development is further speeded up by the continuing advances in additive manufacturing technologies that allow to overcome issues typical of standard processes and to propose tailored designs. However, the design of lattice structures is still challenging since their properties are considerably affected by numerous factors. The present paper aims to propose, discuss, and compare various modeling approaches to describe, understand, and predict the correlations between the mechanical properties and the void volume fraction of different types of lattice structures fabricated by fused deposition modeling 3D printing. Particularly, four approaches are proposed: (i) a simplified analytical model; (ii) a semi-empirical model combining analytical equations with experimental correction factors; (iii) an artificial neural network trained on experimental data; (iv) numerical simulations by finite element analyses. The comparison among the various approaches, and with experimental data, allows to identify the performances, advantages, and disadvantages of each approach, thus giving important guidelines for choosing the right design methodology based on the needs and available data.