pwc-archive/papers-with-abstracts huggingface dataset:
Ultra-wideband Double-Directionally Resolved Channel Measurements of Line-of-Sight Microcellular Scenarios in the Upper Mid-band
Authors: Naveed A. Abbasi, Kelvin Arana, Jorge Gomez-Ponce, Tathagat Pal, Vikram Vasudevan, Atulya Bist, Omer Gokalp Serbetci, Young Han Nam, Charlie Zhang, Andreas F. Molisch
Date: 2024-12-16 00:00:00

The growing demand for higher data rates and expanded bandwidth is driving the exploration of new frequency ranges, including the upper mid-band spectrum (6-24 GHz), which is a promising candidate for future Frequency Range 3 (FR3) applications. This paper presents ultra-wideband double-directional channel measurements in line-of-sight microcellular scenarios within the upper mid-band spectrum (6-18 GHz). Conducted in an urban street canyon environment, these measurements explore key channel characteristics such as power delay profiles, angular power spectra, path loss, delay spread, and angular spread to provide insights essential for robust communication system design. Our results reveal that path loss values for both omni-directional and best beam configurations are lower than free-space predictions due to multipath contributions from the environment. Analysis also indicates a high degree of stability in delay spread and angular spread across the entire band, with small variation between sub-bands.
Alternative Channel Charting Techniques in Cellular Wireless Communications
Authors: Yonghong Jiang, Ender Ayanoglu
Date: 2024-12-14 00:00:00
Tasks: Dimensionality Reduction

We investigate the use of conventional angle of arrival (AoA) algorithms the Bartlett's algorithm, the Minimum Variance Distortion Response (MVDR or Capon) algorithm, and the Minimum Norm algorithm for estimating the AoA $\theta$ together with our previously introduced algorithms linear regression (LR), inverse of the root sum squares of channel coefficients (ISQ), as well as a novel use of the MUSIC algorithm for estimating the distance from the base station, $\rho$ in the context of channel charting. We carry out evaluations in terms of the visual quality of the channel charts, the dimensionality reduction performance measures trustworthiness (TW) and connectivity (CT), as well as the execution time of the algorithms. We find that although the Bartlett's algorithm, MVDR, and Minimum Norm algorithms have sufficiently close performance to techniques we studied earlier, the Minimum Norm algorithm has significantly higher computational complexity than the other two. Previously, we found that the use of the MUSIC algorithm for estimation of both $\theta$ and $\rho$ has a very high performance. In this paper, we investigated and quantified the performance of the Bartlett algorithm in its use for estimating both $\theta$ and $\rho$, similar to the our previously introduced technique of using MUSIC for estimating both.
Representational Drift and Learning-Induced Stabilization in the Olfactory Cortex
Authors: Guillermo B. Morales, Miguel A. Muñoz, Yuhai Tu
Date: 2024-12-18 00:00:00

The brain encodes external stimuli through patterns of neural activity, forming internal representations of the world. Recent experiments show that neural representations for a given stimulus change over time. However, the mechanistic origin for the observed "representational drift" (RD) remains unclear. Here, we propose a biologically-realistic computational model of the piriform cortex to study RD in the mammalian olfactory system by combining two mechanisms for the dynamics of synaptic weights at two separate timescales: spontaneous fluctuations on a scale of days and spike-time dependent plasticity (STDP) on a scale of seconds. Our study shows that, while spontaneous fluctuations in synaptic weights induce RD, STDP-based learning during repeated stimulus presentations can reduce it. Our model quantitatively explains recent experiments on RD in the olfactory system and offers a mechanistic explanation for the emergence of drift and its relation to learning, which may be useful to study RD in other brain regions.
Discover physical concepts and equations with machine learning
Authors: Bao-Bing Li, Yi Gu, Shao-Feng Wu
Date: 2024-12-11 00:00:00

Machine learning can uncover physical concepts or physical equations when prior knowledge from the other is available. However, these two aspects are often intertwined and cannot be discovered independently. We extend SciNet, which is a neural network architecture that simulates the human physical reasoning process for physics discovery, by proposing a model that combines Variational Autoencoders (VAE) with Neural Ordinary Differential Equations (Neural ODEs). This allows us to simultaneously discover physical concepts and governing equations from simulated experimental data across various physical systems. We apply the model to several examples inspired by the history of physics, including Copernicus' heliocentrism, Newton's law of gravity, Schr\"odinger's wave mechanics, and Pauli's spin-magnetic formulation. The results demonstrate that the correct physical theories can emerge in the neural network.
Application of machine learning in grain-related clustering of Laue spots in a polycrystalline energy dispersive Laue pattern
Authors: Amir Tosson, Mohammad Shokr, Mahmoud Al Humaidi, Eduard Mikayelyan, Christian Gutt, Ulrich Pietsch
Date: 2024-12-16 00:00:00
Tasks: Clustering

We address the identification of grain-corresponding Laue reflections in energy dispersive Laue diffraction (EDLD) experiments by formulating it as a clustering problem solvable through unsupervised machine learning (ML). To achieve reliable and efficient identification of grains in a Laue pattern, we employ a combination of clustering algorithms, namely hierarchical clustering (HC) and K-means. These algorithms allow us to group together similar Laue reflections, revealing the underlying grain structure in the diffraction pattern. Additionally, we utilise the elbow method to determine the optimal number of clusters, ensuring accurate results. To evaluate the performance of our proposed method, we conducted experiments using both simulated and experimental datasets obtained from nickel wires. The simulated datasets were generated to mimic the characteristics of real-world EDLD experiments, while the experimental datasets were obtained from actual measurements.
Rare Event Detection in Imbalanced Multi-Class Datasets Using an Optimal MIP-Based Ensemble Weighting Approach
Authors: Georgios Tertytchny, Georgios L. Stavrinides, Maria K. Michael
Date: 2024-12-18 00:00:00
Tasks: Computational Efficiency, Event Detection

To address the challenges of imbalanced multi-class datasets typically used for rare event detection in critical cyber-physical systems, we propose an optimal, efficient, and adaptable mixed integer programming (MIP) ensemble weighting scheme. Our approach leverages the diverse capabilities of the classifier ensemble on a granular per class basis, while optimizing the weights of classifier-class pairs using elastic net regularization for improved robustness and generalization. Additionally, it seamlessly and optimally selects a predefined number of classifiers from a given set. We evaluate and compare our MIP-based method against six well-established weighting schemes, using representative datasets and suitable metrics, under various ensemble sizes. The experimental results reveal that MIP outperforms all existing approaches, achieving an improvement in balanced accuracy ranging from 0.99% to 7.31%, with an overall average of 4.53% across all datasets and ensemble sizes. Furthermore, it attains an overall average increase of 4.63%, 4.60%, and 4.61% in macro-averaged precision, recall, and F1-score, respectively, while maintaining computational efficiency.
Federated Unlearning Model Recovery in Data with Skewed Label Distributions
Authors: Xinrui Yu, Wenbin Pei, Bing Xue, Qiang Zhang
Date: 2024-12-18 00:00:00
Tasks: Denoising, Federated Learning

In federated learning, federated unlearning is a technique that provides clients with a rollback mechanism that allows them to withdraw their data contribution without training from scratch. However, existing research has not considered scenarios with skewed label distributions. Unfortunately, the unlearning of a client with skewed data usually results in biased models and makes it difficult to deliver high-quality service, complicating the recovery process. This paper proposes a recovery method of federated unlearning with skewed label distributions. Specifically, we first adopt a strategy that incorporates oversampling with deep learning to supplement the skewed class data for clients to perform recovery training, therefore enhancing the completeness of their local datasets. Afterward, a density-based denoising method is applied to remove noise from the generated data, further improving the quality of the remaining clients' datasets. Finally, all the remaining clients leverage the enhanced local datasets and engage in iterative training to effectively restore the performance of the unlearning model. Extensive evaluations on commonly used federated learning datasets with varying degrees of skewness show that our method outperforms baseline methods in restoring the performance of the unlearning model, particularly regarding accuracy on the skewed class.
Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction
Authors: Sepideh Maleki, Jan-Christian Huetter, Kangway V. Chuang, David Richmond, Gabriele Scalia, Tommaso Biancalani
Date: 2024-12-18 00:00:00
Tasks: Drug Discovery, Zero-shot Generalization

Predicting transcriptional responses to novel drugs provides a unique opportunity to accelerate biomedical research and advance drug discovery efforts. However, the inherent complexity and high dimensionality of cellular responses, combined with the extremely limited available experimental data, makes the task challenging. In this study, we leverage single-cell foundation models (FMs) pre-trained on tens of millions of single cells, encompassing multiple cell types, states, and disease annotations, to address molecular perturbation prediction. We introduce a drug-conditional adapter that allows efficient fine-tuning by training less than 1% of the original foundation model, thus enabling molecular conditioning while preserving the rich biological representation learned during pre-training. The proposed strategy allows not only the prediction of cellular responses to novel drugs, but also the zero-shot generalization to unseen cell lines. We establish a robust evaluation framework to assess model performance across different generalization tasks, demonstrating state-of-the-art results across all settings, with significant improvements in the few-shot and zero-shot generalization to new cell lines compared to existing baselines.
Learning Causal Transition Matrix for Instance-dependent Label Noise
Authors: Jiahui Li, Tai-Wei Chang, Kun Kuang, Ximing Li, Long Chen, Jun Zhou
Date: 2024-12-18 00:00:00

Noisy labels are both inevitable and problematic in machine learning methods, as they negatively impact models' generalization ability by causing overfitting. In the context of learning with noise, the transition matrix plays a crucial role in the design of statistically consistent algorithms. However, the transition matrix is often considered unidentifiable. One strand of methods typically addresses this problem by assuming that the transition matrix is instance-independent; that is, the probability of mislabeling a particular instance is not influenced by its characteristics or attributes. This assumption is clearly invalid in complex real-world scenarios. To better understand the transition relationship and relax this assumption, we propose to study the data generation process of noisy labels from a causal perspective. We discover that an unobservable latent variable can affect either the instance itself, the label annotation procedure, or both, which complicates the identification of the transition matrix. To address various scenarios, we have unified these observations within a new causal graph. In this graph, the input instance is divided into a noise-resistant component and a noise-sensitive component based on whether they are affected by the latent variable. These two components contribute to identifying the ``causal transition matrix'', which approximates the true transition matrix with theoretical guarantee. In line with this, we have designed a novel training framework that explicitly models this causal relationship and, as a result, achieves a more accurate model for inferring the clean label.
Open-Source Protein Language Models for Function Prediction and Protein Design
Authors: Shivasankaran Vanaja Pandi, Bharath Ramsundar
Date: 2024-12-18 00:00:00
Tasks: Protein Design

Protein language models (PLMs) have shown promise in improving the understanding of protein sequences, contributing to advances in areas such as function prediction and protein engineering. However, training these models from scratch requires significant computational resources, limiting their accessibility. To address this, we integrate a PLM into DeepChem, an open-source framework for computational biology and chemistry, to provide a more accessible platform for protein-related tasks. We evaluate the performance of the integrated model on various protein prediction tasks, showing that it achieves reasonable results across benchmarks. Additionally, we present an exploration of generating plastic-degrading enzyme candidates using the model's embeddings and latent space manipulation techniques. While the results suggest that further refinement is needed, this approach provides a foundation for future work in enzyme design. This study aims to facilitate the use of PLMs in research fields like synthetic biology and environmental sustainability, even for those with limited computational resources.
Rethink the Evaluation Protocol of Model Merging on Classification Task
Authors: Fanshuang Kong, Richong Zhang, Zhijie Nie, Ziqiao Wang
Date: 2024-12-18 00:00:00
Tasks: Classification

Model merging combines multiple fine-tuned models into a single one via parameter fusion, achieving improvements across many tasks. However, in the classification task, we find a misalignment issue between merging outputs and the fine-tuned classifier, which limits its effectiveness. In this paper, we demonstrate the following observations: (1) The embedding quality of the merging outputs is already very high, and the primary reason for the differences in classification performance lies in the misalignment issue. (2) We propose FT-Classifier, a new protocol that fine-tunes an aligned classifier with few-shot samples to alleviate misalignment, enabling better evaluation of merging outputs and improved classification performance. (3) The misalignment is relatively straightforward and can be formulated as an orthogonal transformation. Experiments demonstrate the existence of misalignment and the effectiveness of our FT-Classifier evaluation protocol.
Quantum Machine Learning in Log-based Anomaly Detection: Challenges and Opportunities
Authors: Jiaxing Qi, Chang Zeng, Zhongzhi Luan, Shaohan Huang, Shu Yang, Yao Lu, Bin Han, Hailong Yang, Depei Qian
Date: 2024-12-18 00:00:00
Tasks: Anomaly Detection, Model Selection, Quantum Machine Learning, Specificity

Log-based anomaly detection (LogAD) is the main component of Artificial Intelligence for IT Operations (AIOps), which can detect anomalous that occur during the system on-the-fly. Existing methods commonly extract log sequence features using classical machine learning techniques to identify whether a new sequence is an anomaly or not. However, these classical approaches often require trade-offs between efficiency and accuracy. The advent of quantum machine learning (QML) offers a promising alternative. By transforming parts of classical machine learning computations into parameterized quantum circuits (PQCs), QML can significantly reduce the number of trainable parameters while maintaining accuracy comparable to classical counterparts. In this work, we introduce a unified framework, \ourframework{}, for evaluating QML models in the context of LogAD. This framework incorporates diverse log data, integrated QML models, and comprehensive evaluation metrics. State-of-the-art methods such as DeepLog, LogAnomaly, and LogRobust, along with their quantum-transformed counterparts, are included in our framework.Beyond standard metrics like F1 score, precision, and recall, our evaluation extends to factors critical to QML performance, such as specificity, the number of circuits, circuit design, and quantum state encoding. Using \ourframework{}, we conduct extensive experiments to assess the performance of these models and their quantum counterparts, uncovering valuable insights and paving the way for future research in QML model selection and design for LogAD.
Multi-view Granular-ball Contrastive Clustering
Authors: Peng Su, Shudong Huang, Weihong Ma, Deng Xiong, Jiancheng Lv
Date: 2024-12-18 00:00:00
Tasks: Clustering, Contrastive Learning

Previous multi-view contrastive learning methods typically operate at two scales: instance-level and cluster-level. Instance-level approaches construct positive and negative pairs based on sample correspondences, aiming to bring positive pairs closer and push negative pairs further apart in the latent space. Cluster-level methods focus on calculating cluster assignments for samples under each view and maximize view consensus by reducing distribution discrepancies, e.g., minimizing KL divergence or maximizing mutual information. However, these two types of methods either introduce false negatives, leading to reduced model discriminability, or overlook local structures and cannot measure relationships between clusters across views explicitly. To this end, we propose a method named Multi-view Granular-ball Contrastive Clustering (MGBCC). MGBCC segments the sample set into coarse-grained granular balls, and establishes associations between intra-view and cross-view granular balls. These associations are reinforced in a shared latent space, thereby achieving multi-granularity contrastive learning. Granular balls lie between instances and clusters, naturally preserving the local topological structure of the sample set. We conduct extensive experiments to validate the effectiveness of the proposed method.
Indirect Query Bayesian Optimization with Integrated Feedback
Authors: Mengyan Zhang, Shahine Bouabid, Cheng Soon Ong, Seth Flaxman, Dino Sejdinovic
Date: 2024-12-18 00:00:00
Tasks: Bayesian Optimization, Computational Efficiency

We develop the framework of Indirect Query Bayesian Optimization (IQBO), a new class of Bayesian optimization problems where the integrated feedback is given via a conditional expectation of the unknown function $f$ to be optimized. The underlying conditional distribution can be unknown and learned from data. The goal is to find the global optimum of $f$ by adaptively querying and observing in the space transformed by the conditional distribution. This is motivated by real-world applications where one cannot access direct feedback due to privacy, hardware or computational constraints. We propose the Conditional Max-Value Entropy Search (CMES) acquisition function to address this novel setting, and propose a hierarchical search algorithm to address the multi-resolution setting and improve the computational efficiency. We show regret bounds for our proposed methods and demonstrate the effectiveness of our approaches on simulated optimization tasks.
TAUDiff: Improving statistical downscaling for extreme weather events using generative diffusion models
Authors: Rahul Sundar, Nishant Parashar, Antoine Blanchard, Boyko Dodov
Date: 2024-12-18 00:00:00

Deterministic regression-based downscaling models for climate variables often suffer from spectral bias, which can be mitigated by generative models like diffusion models. To enable efficient and reliable simulation of extreme weather events, it is crucial to achieve rapid turnaround, dynamical consistency, and accurate spatio-temporal spectral recovery. We propose an efficient correction diffusion model, TAUDiff, that combines a deterministic spatio-temporal model for mean field downscaling with a smaller generative diffusion model for recovering the fine-scale stochastic features. We demonstrate the efficacy of this approach on downscaling atmospheric wind velocity fields obtained from coarse GCM simulations. Our approach can not only ensure quicker simulation of extreme events but also reduce overall carbon footprint due to low inference times.
Personalized Clustering via Targeted Representation Learning
Authors: Xiwen Geng, Suyun Zhao, Yixin Yu, Borui Peng, Pan Du, Hong Chen, Cuiping Li, Mengdie Wang
Date: 2024-12-18 00:00:00
Tasks: Clustering, Representation Learning

Clustering traditionally aims to reveal a natural grouping structure within unlabeled data. However, this structure may not always align with users' preferences. In this paper, we propose a personalized clustering method that explicitly performs targeted representation learning by interacting with users via modicum task information (e.g., $\textit{must-link}$ or $\textit{cannot-link}$ pairs) to guide the clustering direction. We query users with the most informative pairs, i.e., those pairs most hard to cluster and those most easy to miscluster, to facilitate the representation learning in terms of the clustering preference. Moreover, by exploiting attention mechanism, the targeted representation is learned and augmented. By leveraging the targeted representation and constrained contrastive loss as well, personalized clustering is obtained. Theoretically, we verify that the risk of personalized clustering is tightly bounded, guaranteeing that active queries to users do mitigate the clustering risk. Experimentally, extensive results show that our method performs well across different clustering tasks and datasets, even when only a limited number of queries are available.
Splitting criteria for ordinal decision trees: an experimental study
Authors: Rafael Ayllón-Gavilán, Francisco José Martínez-Estudillo, David Guijo-Rubio, César Hervás-Martínez, Pedro Antonio Gutiérrez
Date: 2024-12-18 00:00:00
Tasks: Classification, Ordinal Classification

Ordinal Classification (OC) is a machine learning field that addresses classification tasks where the labels exhibit a natural order. Unlike nominal classification, which treats all classes as equally distinct, OC takes the ordinal relationship into account, producing more accurate and relevant results. This is particularly critical in applications where the magnitude of classification errors has implications. Despite this, OC problems are often tackled using nominal methods, leading to suboptimal solutions. Although decision trees are one of the most popular classification approaches, ordinal tree-based approaches have received less attention when compared to other classifiers. This work conducts an experimental study of tree-based methodologies specifically designed to capture ordinal relationships. A comprehensive survey of ordinal splitting criteria is provided, standardising the notations used in the literature for clarity. Three ordinal splitting criteria, Ordinal Gini (OGini), Weighted Information Gain (WIG), and Ranking Impurity (RI), are compared to the nominal counterparts of the first two (Gini and information gain), by incorporating them into a decision tree classifier. An extensive repository considering 45 publicly available OC datasets is presented, supporting the first experimental comparison of ordinal and nominal splitting criteria using well-known OC evaluation metrics. Statistical analysis of the results highlights OGini as the most effective ordinal splitting criterion to date. Source code, datasets, and results are made available to the research community.