pwc-archive/papers-with-abstracts huggingface dataset:
Large Language Model Aided QoS Prediction for Service Recommendation
Authors: Huiying Liu, Zekun Zhang, Honghao Li, Qilin Wu, Yiwen Zhang
Date: 2024-08-05 00:00:00
Tasks: Descriptive, Language Modeling, Language Modelling, Large Language Model, Prediction

Large language models (LLMs) have seen rapid improvement in the recent years, and have been used in a wider range of applications. After being trained on large text corpus, LLMs obtain the capability of extracting rich features from textual data. Such capability is potentially useful for the web service recommendation task, where the web users and services have intrinsic attributes that can be described using natural language sentences and are useful for recommendation. In this paper, we explore the possibility and practicality of using LLMs for web service recommendation. We propose the large language model aided QoS prediction (llmQoS) model, which use LLMs to extract useful information from attributes of web users and services via descriptive sentences. This information is then used in combination with the QoS values of historical interactions of users and services, to predict QoS values for any given user-service pair. On the WSDream dataset, llmQoS is shown to overcome the data sparsity issue inherent to the QoS prediction problem, and outperforms comparable baseline models consistently.
Contrastive Learning and Abstract Concepts: The Case of Natural Numbers
Authors: Daniel N. Nissani
Date: 2024-08-05 00:00:00
Tasks: Contrastive Learning

Contrastive Learning (CL) has been successfully applied to classification and other downstream tasks related to concrete concepts, such as objects contained in the ImageNet dataset. No attempts seem to have been made so far in applying this promising scheme to more abstract entities. A prominent example of these could be the concept of (discrete) Quantity. CL can be frequently interpreted as a self-supervised scheme guided by some profound and ubiquitous conservation principle (e.g. conservation of identity in object classification tasks). In this introductory work we apply a suitable conservation principle to the semi-abstract concept of natural numbers by which discrete quantities can be estimated or predicted. We experimentally show, by means of a toy problem, that contrastive learning can be trained to count at a glance with high accuracy both at human as well as at super-human ranges.. We compare this with the results of a trained-to-count at a glance supervised learning (SL) neural network scheme of similar architecture. We show that both schemes exhibit similar good performance on baseline experiments, where the distributions of the training and testing stages are equal. Importantly, we demonstrate that in some generalization scenarios, where training and testing distributions differ, CL boasts more robust and much better error performance.
One-Shot Collaborative Data Distillation
Authors: William Holland, Chandra Thapa, Sarah Ali Siddiqui, Wei Shao, Seyit Camtepe
Date: 2024-08-05 00:00:00
Tasks: One-Shot Learning

Large machine-learning training datasets can be distilled into small collections of informative synthetic data samples. These synthetic sets support efficient model learning and reduce the communication cost of data sharing. Thus, high-fidelity distilled data can support the efficient deployment of machine learning applications in distributed network environments. A naive way to construct a synthetic set in a distributed environment is to allow each client to perform local data distillation and to merge local distillations at a central server. However, the quality of the resulting set is impaired by heterogeneity in the distributions of the local data held by clients. To overcome this challenge, we introduce the first collaborative data distillation technique, called CollabDM, which captures the global distribution of the data and requires only a single round of communication between client and server. Our method outperforms the state-of-the-art one-shot learning method on skewed data in distributed learning environments. We also show the promising practical benefits of our method when applied to attack detection in 5G networks.
DRFormer: Multi-Scale Transformer Utilizing Diverse Receptive Fields for Long Time-Series Forecasting
Authors: Ruixin Ding, Yuqi Chen, Yu-Ting Lan, Wei zhang
Date: 2024-08-05 00:00:00
Tasks: Position, Sparse Learning, Time Series, Time Series Forecasting, Traffic Prediction

Long-term time series forecasting (LTSF) has been widely applied in finance, traffic prediction, and other domains. Recently, patch-based transformers have emerged as a promising approach, segmenting data into sub-level patches that serve as input tokens. However, existing methods mostly rely on predetermined patch lengths, necessitating expert knowledge and posing challenges in capturing diverse characteristics across various scales. Moreover, time series data exhibit diverse variations and fluctuations across different temporal scales, which traditional approaches struggle to model effectively. In this paper, we propose a dynamic tokenizer with a dynamic sparse learning algorithm to capture diverse receptive fields and sparse patterns of time series data. In order to build hierarchical receptive fields, we develop a multi-scale Transformer model, coupled with multi-scale sequence extraction, capable of capturing multi-resolution features. Additionally, we introduce a group-aware rotary position encoding technique to enhance intra- and inter-group position awareness among representations across different temporal scales. Our proposed model, named DRFormer, is evaluated on various real-world datasets, and experimental results demonstrate its superiority compared to existing methods. Our code is available at: https://github.com/ruixindingECNU/DRFormer.
Hardware Aware Ensemble Selection for Balancing Predictive Accuracy and Cost
Authors: Jannis Maier, Felix Möller, Lennart Purucker
Date: 2024-08-05 00:00:00
Tasks: AutoML, Diversity, Model Selection

Automated Machine Learning (AutoML) significantly simplifies the deployment of machine learning models by automating tasks from data preprocessing to model selection to ensembling. AutoML systems for tabular data often employ post hoc ensembling, where multiple models are combined to improve predictive accuracy. This typically results in longer inference times, a major limitation in practical deployments. Addressing this, we introduce a hardware-aware ensemble selection approach that integrates inference time into post hoc ensembling. By leveraging an existing framework for ensemble selection with quality diversity optimization, our method evaluates ensemble candidates for their predictive accuracy and hardware efficiency. This dual focus allows for a balanced consideration of accuracy and operational efficiency. Thus, our approach enables practitioners to choose from a Pareto front of accurate and efficient ensembles. Our evaluation using 83 classification datasets shows that our approach sustains competitive accuracy and can significantly improve ensembles' operational efficiency. The results of this study provide a foundation for extending these principles to additional hardware constraints, setting the stage for the development of more resource-efficient AutoML systems.
Heart Rate and its Variability from Short-term ECG Recordings as Biomarkers for Detecting Mild Cognitive Impairment in Indian Population
Authors: Anjo Xavier, Sneha Noble, Justin Joseph, Thomas Gregor Issac
Date: 2024-08-05 00:00:00
Tasks: Heart Rate Variability

Alterations in Heart Rate (HR) and Heart Rate Variability (HRV) can reflect autonomic dysfunction associated with neurodegeneration. We investigate the influence of Mild Cognitive Impairment (MCI) on HR and its variability measures in the Indian population by designing a complete signal processing pipeline to detect the R-wave peaks and compute HR and HRV features from ECG recordings of 10 seconds, for point-of-care applications. The study cohort involves 297 urban participants, among which 48.48% are male and 51.51% are female. From the Addenbrooke's Cognitive Examination-III (ACE-III), MCI is detected in 19.19% of participants and the rest, 80.8% of them are cognitively healthy. Statistical features like central tendency (mean and root mean square (RMS) of the Normal-to-Normal (NN) intervals) and dispersion (standard deviation (SD) of all NN intervals (SDNN) and root mean square of successive differences of NN intervals (RMSSD)) of beat-to-beat intervals are computed. The Wilcoxon rank sum test reveals that mean of NN intervals (p = 0.0021), the RMS of NN intervals (p = 0.0014), the SDNN (p = 0.0192) and the RMSSD (p = 0.0206) values differ significantly between MCI and non-MCI classes, for a level of significance, 0.05. Machine learning classifiers like, Support Vector Machine (SVM), Discriminant Analysis (DA) and Naive Bayes (NB) driven by mean NN intervals, RMS, SDNN and RMSSD, show a high accuracy of 80.80% on each individual feature input. Individuals with MCI are observed to have comparatively higher HR than healthy subjects. HR and its variability can be considered as potential biomarkers for detecting MCI.
Backward Compatibility in Attributive Explanation and Enhanced Model Training Method
Authors: Ryuta Matsuno
Date: 2024-08-05 00:00:00

Model update is a crucial process in the operation of ML/AI systems. While updating a model generally enhances the average prediction performance, it also significantly impacts the explanations of predictions. In real-world applications, even minor changes in explanations can have detrimental consequences. To tackle this issue, this paper introduces BCX, a quantitative metric that evaluates the backward compatibility of feature attribution explanations between pre- and post-update models. BCX utilizes practical agreement metrics to calculate the average agreement between the explanations of pre- and post-update models, specifically among samples on which both models accurately predict. In addition, we propose BCXR, a BCX-aware model training method by designing surrogate losses which theoretically lower bounds agreement scores. Furthermore, we present a universal variant of BCXR that improves all agreement metrics, utilizing L2 distance among the explanations of the models. To validate our approach, we conducted experiments on eight real-world datasets, demonstrating that BCXR achieves superior trade-offs between predictive performances and BCX scores, showcasing the effectiveness of our BCXR methods.
A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models
Authors: Gen Li, Yuting Wei, Yuejie Chi, Yuxin Chen
Date: 2024-08-05 00:00:00

Diffusion models, which convert noise into new data instances by learning to reverse a diffusion process, have become a cornerstone in contemporary generative modeling. In this work, we develop non-asymptotic convergence theory for a popular diffusion-based sampler (i.e., the probability flow ODE sampler) in discrete time, assuming access to $\ell_2$-accurate estimates of the (Stein) score functions. For distributions in $\mathbb{R}^d$, we prove that $d/\varepsilon$ iterations -- modulo some logarithmic and lower-order terms -- are sufficient to approximate the target distribution to within $\varepsilon$ total-variation distance. This is the first result establishing nearly linear dimension-dependency (in $d$) for the probability flow ODE sampler. Imposing only minimal assumptions on the target data distribution (e.g., no smoothness assumption is imposed), our results also characterize how $\ell_2$ score estimation errors affect the quality of the data generation processes. In contrast to prior works, our theory is developed based on an elementary yet versatile non-asymptotic approach without the need of resorting to SDE and ODE toolboxes.
Machine Learning Applications in Medical Prognostics: A Comprehensive Review
Authors: Michael Fascia
Date: 2024-08-05 00:00:00
Tasks: Disease Prediction, Transfer Learning

Machine learning (ML) has revolutionized medical prognostics by integrating advanced algorithms with clinical data to enhance disease prediction, risk assessment, and patient outcome forecasting. This comprehensive review critically examines the application of various ML techniques in medical prognostics, focusing on their efficacy, challenges, and future directions. The methodologies discussed include Random Forest (RF) for sepsis prediction, logistic regression for cardiovascular risk assessment, Convolutional Neural Networks (CNNs) for cancer detection, and Long Short-Term Memory (LSTM) networks for predicting clinical deterioration. RF models demonstrate robust performance in handling high-dimensional data and capturing non-linear relationships, making them particularly effective for sepsis prediction. Logistic regression remains valuable for its interpretability and ease of use in cardiovascular risk assessment. CNNs have shown exceptional accuracy in cancer detection, leveraging their ability to learn complex visual patterns from medical imaging. LSTM networks excel in analyzing temporal data, providing accurate predictions of clinical deterioration. The review highlights the strengths and limitations of each technique, the importance of model interpretability, and the challenges of data quality and privacy. Future research directions include the integration of multi-modal data sources, the application of transfer learning, and the development of continuous learning systems. These advancements aim to enhance the predictive power and clinical applicability of ML models, ultimately improving patient outcomes in healthcare settings.
Exploiting Hankel-Toeplitz Structures for Fast Computation of Kernel Precision Matrices
Authors: Frida Viset, Anton Kullberg, Frederiek Wesel, Arno Solin
Date: 2024-08-05 00:00:00
Tasks: Hyperparameter Optimization

The Hilbert-space Gaussian Process (HGP) approach offers a hyperparameter-independent basis function approximation for speeding up Gaussian Process (GP) inference by projecting the GP onto M basis functions. These properties result in a favorable data-independent $\mathcal{O}(M^3)$ computational complexity during hyperparameter optimization but require a dominating one-time precomputation of the precision matrix costing $\mathcal{O}(NM^2)$ operations. In this paper, we lower this dominating computational complexity to $\mathcal{O}(NM)$ with no additional approximations. We can do this because we realize that the precision matrix can be split into a sum of Hankel-Toeplitz matrices, each having $\mathcal{O}(M)$ unique entries. Based on this realization we propose computing only these unique entries at $\mathcal{O}(NM)$ costs. Further, we develop two theorems that prescribe sufficient conditions for the complexity reduction to hold generally for a wide range of other approximate GP models, such as the Variational Fourier Feature (VFF) approach. The two theorems do this with no assumptions on the data and no additional approximations of the GP models themselves. Thus, our contribution provides a pure speed-up of several existing, widely used, GP approximations, without further approximations.
Active Sensing of Knee Osteoarthritis Progression with Reinforcement Learning
Authors: Khanh Nguyen, Huy Hoang Nguyen, Egor Panfilov, Aleksei Tiulpin
Date: 2024-08-05 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Osteoarthritis (OA) is the most common musculoskeletal disease, which has no cure. Knee OA (KOA) is one of the highest causes of disability worldwide, and it costs billions of United States dollars to the global community. Prediction of KOA progression has been of high interest to the community for years, as it can advance treatment development through more efficient clinical trials and improve patient outcomes through more efficient healthcare utilization. Existing approaches for predicting KOA, however, are predominantly static, i.e. consider data from a single time point to predict progression many years into the future, and knee level, i.e. consider progression in a single joint only. Due to these and related reasons, these methods fail to deliver the level of predictive performance, which is sufficient to result in cost savings and better patient outcomes. Collecting extensive data from all patients on a regular basis could address the issue, but it is limited by the high cost at a population level. In this work, we propose to go beyond static prediction models in OA, and bring a novel Active Sensing (AS) approach, designed to dynamically follow up patients with the objective of maximizing the number of informative data acquisitions, while minimizing their total cost over a period of time. Our approach is based on Reinforcement Learning (RL), and it leverages a novel reward function designed specifically for AS of disease progression in more than one part of a human body. Our method is end-to-end, relies on multi-modal Deep Learning, and requires no human input at inference time. Throughout an exhaustive experimental evaluation, we show that using RL can provide a higher monetary benefit when compared to state-of-the-art baselines.
Strategic Federated Learning: Application to Smart Meter Data Clustering
Authors: Hassan Mohamad, Chao Zhang, Samson Lasaulce, Vineeth S Varma, Mérouane Debbah, Mounir Ghogho
Date: 2024-08-05 00:00:00
Tasks: Clustering, Federated Learning, Scheduling

Federated learning (FL) involves several clients that share with a fusion center (FC), the model each client has trained with its own data. Conventional FL, which can be interpreted as an estimation or distortion-based approach, ignores the final use of model information (MI) by the FC and the other clients. In this paper, we introduce a novel FL framework in which the FC uses an aggregate version of the MI to make decisions that affect the client's utility functions. Clients cannot choose the decisions and can only use the MI reported to the FC to maximize their utility. Depending on the alignment between the client and FC utilities, the client may have an individual interest in adding strategic noise to the model. This general framework is stated and specialized to the case of clustering, in which noisy cluster representative information is reported. This is applied to the problem of power consumption scheduling. In this context, utility non-alignment occurs, for instance, when the client wants to consume when the price of electricity is low, whereas the FC wants the consumption to occur when the total power is the lowest. This is illustrated with aggregated real data from Ausgrid \cite{ausgrid}. Our numerical analysis clearly shows that the client can increase his utility by adding noise to the model reported to the FC. Corresponding results and source codes can be downloaded from \cite{source-code}.
Terracorder: Sense Long and Prosper
Authors: Josh Millar, Sarab Sethi, Hamed Haddadi, Anil Madhavapeddy
Date: 2024-08-05 00:00:00

In-situ sensing devices need to be deployed in remote environments for long periods of time; minimizing their power consumption is vital for maximising both their operational lifetime and coverage. We introduce Terracorder -- a versatile multi-sensor device -- and showcase its exceptionally low power consumption using an on-device reinforcement learning scheduler. We prototype a unique device setup for biodiversity monitoring and compare its battery life using our scheduler against a number of fixed schedules; the scheduler captures more than 80% of events at less than 50% of the number of activations of the best-performing fixed schedule. We then explore how a collaborative scheduler can maximise the useful operation of a network of devices, improving overall network power consumption and robustness.
Enhancing Heterogeneous Knowledge Graph Completion with a Novel GAT-based Approach
Date: 2024-08-05 00:00:00

Knowledge graphs (KGs) play a vital role in enhancing search results and recommendation systems. With the rapid increase in the size of the KGs, they are becoming inaccuracy and incomplete. This problem can be solved by the knowledge graph completion methods, of which graph attention network (GAT)-based methods stand out since their superior performance. However, existing GAT-based knowledge graph completion methods often suffer from overfitting issues when dealing with heterogeneous knowledge graphs, primarily due to the unbalanced number of samples. Additionally, these methods demonstrate poor performance in predicting the tail (head) entity that shares the same relation and head (tail) entity with others. To solve these problems, we propose GATH, a novel GAT-based method designed for Heterogeneous KGs. GATH incorporates two separate attention network modules that work synergistically to predict the missing entities. We also introduce novel encoding and feature transformation approaches, enabling the robust performance of GATH in scenarios with imbalanced samples. Comprehensive experiments are conducted to evaluate the GATH's performance. Compared with the existing SOTA GAT-based model on Hits@10 and MRR metrics, our model improves performance by 5.2% and 5.2% on the FB15K-237 dataset, and by 4.5% and 14.6% on the WN18RR dataset, respectively.
LMEMs for post-hoc analysis of HPO Benchmarking
Authors: Anton Geburek, Neeratyoy Mallik, Danny Stoll, Xavier Bouthillier, Frank Hutter
Date: 2024-08-05 00:00:00
Tasks: Benchmarking, Hyperparameter Optimization

The importance of tuning hyperparameters in Machine Learning (ML) and Deep Learning (DL) is established through empirical research and applications, evident from the increase in new hyperparameter optimization (HPO) algorithms and benchmarks steadily added by the community. However, current benchmarking practices using averaged performance across many datasets may obscure key differences between HPO methods, especially for pairwise comparisons. In this work, we apply Linear Mixed-Effect Models-based (LMEMs) significance testing for post-hoc analysis of HPO benchmarking runs. LMEMs allow flexible and expressive modeling on the entire experiment data, including information such as benchmark meta-features, offering deeper insights than current analysis practices. We demonstrate this through a case study on the PriorBand paper's experiment data to find insights not reported in the original work.
Process-constrained batch Bayesian approaches for yield optimization in multi-reactor systems
Authors: Markus Grimm, Sébastien Paul, Pierre Chainais
Date: 2024-08-05 00:00:00
Tasks: Bayesian Optimization, Thompson Sampling

The optimization of yields in multi-reactor systems, which are advanced tools in heterogeneous catalysis research, presents a significant challenge due to hierarchical technical constraints. To this respect, this work introduces a novel approach called process-constrained batch Bayesian optimization via Thompson sampling (pc-BO-TS) and its generalized hierarchical extension (hpc-BO-TS). This method, tailored for the efficiency demands in multi-reactor systems, integrates experimental constraints and balances between exploration and exploitation in a sequential batch optimization strategy. It offers an improvement over other Bayesian optimization methods. The performance of pc-BO-TS and hpc-BO-TS is validated in synthetic cases as well as in a realistic scenario based on data obtained from high-throughput experiments done on a multi-reactor system available in the REALCAT platform. The proposed methods often outperform other sequential Bayesian optimizations and existing process-constrained batch Bayesian optimization methods. This work proposes a novel approach to optimize the yield of a reaction in a multi-reactor system, marking a significant step forward in digital catalysis and generally in optimization methods for chemical engineering.