pwc-archive/papers-with-abstracts huggingface dataset:
Inverse Particle Filter
Authors: Himali Singh, Arpan Chattopadhyay, Kumar Vijay Mishra
Date: 2024-07-23 00:00:00

In cognitive systems, recent emphasis has been placed on studying the cognitive processes of the subject whose behavior was the primary focus of the system's cognitive response. This approach, known as inverse cognition, arises in counter-adversarial applications and has motivated the development of inverse Bayesian filters. In this context, a cognitive adversary, such as a radar, uses a forward Bayesian filter to track its target of interest. An inverse filter is then employed to infer the adversary's estimate of the target's or defender's state. Previous studies have addressed this inverse filtering problem by introducing methods like the inverse Kalman filter (KF), inverse extended KF, and inverse unscented KF. However, these filters typically assume additive Gaussian noise models and/or rely on local approximations of non-linear dynamics at the state estimates, limiting their practical application. In contrast, this paper adopts a global filtering approach and presents the development of an inverse particle filter (I-PF). The particle filter framework employs Monte Carlo (MC) methods to approximate arbitrary posterior distributions. Moreover, under mild system-level conditions, the proposed I-PF demonstrates convergence to the optimal inverse filter. Additionally, we propose the differentiable I-PF to address scenarios where system information is unknown to the defender. Using the recursive Cramer-Rao lower bound and non-credibility index (NCI), our numerical experiments for different systems demonstrate the estimation performance and time complexity of the proposed filter.
A Kernel-Based Conditional Two-Sample Test Using Nearest Neighbors (with Applications to Calibration, Regression Curves, and Simulation-Based Inference)
Authors: Anirban Chatterjee, Ziang Niu, Bhaswar B. Bhattacharya
Date: 2024-07-23 00:00:00

In this paper we introduce a kernel-based measure for detecting differences between two conditional distributions. Using the `kernel trick' and nearest-neighbor graphs, we propose a consistent estimate of this measure which can be computed in nearly linear time (for a fixed number of nearest neighbors). Moreover, when the two conditional distributions are the same, the estimate has a Gaussian limit and its asymptotic variance has a simple form that can be easily estimated from the data. The resulting test attains precise asymptotic level and is universally consistent for detecting differences between two conditional distributions. We also provide a resampling based test using our estimate that applies to the conditional goodness-of-fit problem, which controls Type I error in finite samples and is asymptotically consistent with only a finite number of resamples. A method to de-randomize the resampling test is also presented. The proposed methods can be readily applied to a broad range of problems, ranging from classical nonparametric statistics to modern machine learning. Specifically, we explore three applications: testing model calibration, regression curve evaluation, and validation of emulator models in simulation-based inference. We illustrate the superior performance of our method for these tasks, both in simulations as well as on real data. In particular, we apply our method to (1) assess the calibration of neural network models trained on the CIFAR-10 dataset, (2) compare regression functions for wind power generation across two different turbines, and (3) validate emulator models on benchmark examples with intractable posteriors and for generating synthetic `redshift' associated with galaxy images.
Neural information field filter
Authors: Kairui Hao, Ilias Bilionis
Date: 2024-07-23 00:00:00
Tasks: parameter estimation, Variational Inference

We introduce neural information field filter, a Bayesian state and parameter estimation method for high-dimensional nonlinear dynamical systems given large measurement datasets. Solving such a problem using traditional methods, such as Kalman and particle filters, is computationally expensive. Information field theory is a Bayesian approach that can efficiently reconstruct dynamical model state paths and calibrate model parameters from noisy measurement data. To apply the method, we parameterize the time evolution state path using the span of a finite linear basis. The existing method has to reparameterize the state path by initial states to satisfy the initial condition. Designing an expressive yet simple linear basis before knowing the true state path is crucial for inference accuracy but challenging. Moreover, reparameterizing the state path using the initial state is easy to perform for a linear basis, but is nontrivial for more complex and expressive function parameterizations, such as neural networks. The objective of this paper is to simplify and enrich the class of state path parameterizations using neural networks for the information field theory approach. To this end, we propose a generalized physics-informed conditional prior using an auxiliary initial state. We show the existing reparameterization is a special case. We parameterize the state path using a residual neural network that consists of a linear basis function and a Fourier encoding fully connected neural network residual function. The residual function aims to correct the error of the linear basis function. To sample from the intractable posterior distribution, we develop an optimization algorithm, nested stochastic variational inference, and a sampling algorithm, nested preconditioned stochastic gradient Langevin dynamics. A series of numerical and experimental examples verify and validate the proposed method.
Sparse outlier-robust PCA for multi-source data
Authors: Patricia Puchhammer, Ines Wilms, Peter Filzmoser
Date: 2024-07-23 00:00:00

Sparse and outlier-robust Principal Component Analysis (PCA) has been a very active field of research recently. Yet, most existing methods apply PCA to a single dataset whereas multi-source data-i.e. multiple related datasets requiring joint analysis-arise across many scientific areas. We introduce a novel PCA methodology that simultaneously (i) selects important features, (ii) allows for the detection of global sparse patterns across multiple data sources as well as local source-specific patterns, and (iii) is resistant to outliers. To this end, we develop a regularization problem with a penalty that accommodates global-local structured sparsity patterns, and where the ssMRCD estimator is used as plug-in to permit joint outlier-robust analysis across multiple data sources. We provide an efficient implementation of our proposal via the Alternating Direction Method of Multiplier and illustrate its practical advantages in simulation and in applications.
Decoding Digital Influence: The Role of Social Media Behavior in Scientific Stratification Through Logistic Attribution Method
Authors: Yang Yue
Date: 2024-07-23 00:00:00
Tasks: Experimental Design, Sociology

Scientific social stratification is a classic theme in the sociology of science. The deep integration of social media has bridged the gap between scientometrics and sociology of science. This study comprehensively analyzes the impact of social media on scientific stratification and mobility, delving into the complex interplay between academic status and social media activity in the digital age. [Research Method] Innovatively, this paper employs An Explainable Logistic Attribution Analysis from a meso-level perspective to explore the correlation between social media behaviors and scientific social stratification. It examines the impact of scientists' use of social media in the digital age on scientific stratification and mobility, uniquely combining statistical methods with machine learning. This fusion effectively integrates hypothesis testing with a substantive interpretation of the contribution of independent variables to the model. [Research Conclusion] Empirical evidence demonstrates that social media promotes stratification and mobility within the scientific community, revealing a nuanced and non-linear facilitation mechanism. Social media activities positively impact scientists' status within the scientific social hierarchy to a certain extent, but beyond a specific threshold, this impact turns negative. It shows that the advent of social media has opened new channels for academic influence, transcending the limitations of traditional academic publishing, and prompting changes in scientific stratification. Additionally, the study acknowledges the limitations of its experimental design and suggests future research directions.
Estimating Distributional Treatment Effects in Randomized Experiments: Machine Learning for Variance Reduction
Authors: Undral Byambadalai, Tatsushi Oka, Shota Yasui
Date: 2024-07-22 00:00:00
Tasks: regression, valid

We propose a novel regression adjustment method designed for estimating distributional treatment effect parameters in randomized experiments. Randomized experiments have been extensively used to estimate treatment effects in various scientific fields. However, to gain deeper insights, it is essential to estimate distributional treatment effects rather than relying solely on average effects. Our approach incorporates pre-treatment covariates into a distributional regression framework, utilizing machine learning techniques to improve the precision of distributional treatment effect estimators. The proposed approach can be readily implemented with off-the-shelf machine learning methods and remains valid as long as the nuisance components are reasonably well estimated. Also, we establish the asymptotic properties of the proposed estimator and present a uniformly valid inference method. Through simulation results and real data analysis, we demonstrate the effectiveness of integrating machine learning techniques in reducing the variance of distributional treatment effect estimators in finite samples.
Ensemble quantile-based deep learning framework for streamflow and flood prediction in Australian catchments
Authors: Rohitash Chandra, Arpit Kapoor, Siddharth Khedkar, Jim Ng, R. Willem Vervoort
Date: 2024-07-20 00:00:00
Tasks: Deep Learning, Prediction, quantile regression, Time Series, Time Series Prediction

In recent years, climate extremes such as floods have created significant environmental and economic hazards for Australia. Deep learning methods have been promising for predicting extreme climate events; however, large flooding events present a critical challenge due to factors such as model calibration and missing data. We present an ensemble quantile-based deep learning framework that addresses large-scale streamflow forecasts using quantile regression for uncertainty projections in prediction. We evaluate selected univariate and multivariate deep learning models and catchment strategies. Furthermore, we implement a multistep time-series prediction model using the CAMELS dataset for selected catchments across Australia. The ensemble model employs a set of quantile deep learning models for streamflow determined by historical streamflow data. We utilise the streamflow prediction and obtain flood probability using flood frequency analysis and compare it with historical flooding events for selected catchments. Our results demonstrate notable efficacy and uncertainties in streamflow forecasts with varied catchment properties. Our flood probability estimates show good accuracy in capturing the historical floods from the selected catchments. This underscores the potential for our deep learning framework to revolutionise flood forecasting across diverse regions and be implemented as an early warning system.
A Survey on Differential Privacy for SpatioTemporal Data in Transportation Research
Authors: Rahul Bhadani
Date: 2024-07-18 00:00:00

With low-cost computing devices, improved sensor technology, and the proliferation of data-driven algorithms, we have more data than we know what to do with. In transportation, we are seeing a surge in spatiotemporal data collection. At the same time, concerns over user privacy have led to research on differential privacy in applied settings. In this paper, we look at some recent developments in differential privacy in the context of spatiotemporal data. Spatiotemporal data contain not only features about users but also the geographical locations of their frequent visits. Hence, the public release of such data carries extreme risks. To address the need for such data in research and inference without exposing private information, significant work has been proposed. This survey paper aims to summarize these efforts and provide a review of differential privacy mechanisms and related software. We also discuss related work in transportation where such mechanisms have been applied. Furthermore, we address the challenges in the deployment and mass adoption of differential privacy in transportation spatiotemporal data for downstream analyses.
Sparks of Quantum Advantage and Rapid Retraining in Machine Learning
Authors: William Troy
Date: 2024-07-22 00:00:00
Tasks: Kolmogorov-Arnold Networks

The advent of quantum computing holds the potential to revolutionize various fields by solving complex problems more efficiently than classical computers. Despite this promise, practical quantum advantage is hindered by current hardware limitations, notably the small number of qubits and high noise levels. In this study, we leverage adiabatic quantum computers to optimize Kolmogorov-Arnold Networks, a powerful neural network architecture for representing complex functions with minimal parameters. By modifying the network to use Bezier curves as the basis functions and formulating the optimization problem into a Quadratic Unconstrained Binary Optimization problem, we create a fixed-sized solution space, independent of the number of training samples. This strategy allows for the optimization of an entire neural network in a single training iteration in which, due to order of operations, a majority of the processing is done using a collapsed version of the training dataset. This inherently creates extremely fast training speeds, which are validated experimentally, compared to classical optimizers including Adam, Stochastic Gradient Descent, Adaptive Gradient, and simulated annealing. Additionally, we introduce a novel rapid retraining capability, enabling the network to be retrained with new data without reprocessing old samples, thus enhancing learning efficiency in dynamic environments. Experiments on retraining demonstrate a hundred times speed up using adiabatic quantum computing based optimization compared to that of the gradient descent based optimizers, with theoretical models allowing this speed up to be much larger! Our findings suggest that with further advancements in quantum hardware and algorithm optimization, quantum-optimized machine learning models could have broad applications across various domains, with initial focus on rapid retraining.
Are Graphs and GCNs necessary for short-term metro ridership forecasting?
Authors: Qiong Yang, Xianghua Xu, Zihang Wang, Juan Yu, Xiaodong Hu
Date: 2024-06-06 00:00:00
Proceeding: Expert Systems with Applications 2024 6
Tasks: Decoder, Prediction, Traffic Prediction

Short-term metro ridership prediction is of great significance to efficient and economic operation of Urban Rail Transit (URT) systems. With the popularity of Graph Convolution Networks (GCN) and Transformers, the recent notable metro ridership forecasting methods are GCN-based and Transformer-based models. However, existing methods face the following drawbacks. First, GCN-based models fail to effectively capture global spatial correlations which are significant for accurate prediction. Second, Transformer-based models are prone to loss temporal information due to the permutation-invariant and anti-order properties of the self-attention which they used for capturing temporal correlations. To overcome the drawbacks, we propose a novel sequence-to-sequence metro ridership prediction model, named SDT-GRU, with Stacked DT-GRU layers as both encoder and decoder. The core component of our model is DT-GRU, which integrates Dual-branch Transformer decoder into the GRU to effectively capture global spatial correlations and temporal correlations with Transformer decoder and GRU, separately. In particular, the DT-GRU module uses one branch Transformer encoder layer to capture spatial correlations within the same timestamp, and adopts another Transformer encoder layer to implicitly capture spatio-temporal correlations among previous timestamps. Then, outputs of the two Transformer encoder layers are fed into a GRU layer to capturing spatio-temporal patterns. To evaluate the effectiveness of the proposed SDT-GRU, we conduct comprehensive experiments on three real-world metro ridership datasets from Beijing, Shanghai and Hangzhou. Experimental results demonstrate that our SDT-GRU achieves better prediction performance than the state-of-the-art baselines.
Prediction of lipid nanoparticles for mRNA vaccines by the machine learning algorithm
Authors: Wei Wang, Shuo Feng, Zhuyifan Ye, Hanlu Gao, Jinzhong Lin, Defang Ouyang
Date: 2021-11-28 00:00:00
Proceeding: Acta Pharmaceutica Sinica B 2021 11

Lipid nanoparticle (LNP) is commonly used to deliver mRNA vaccines.  Currently, LNP optimization primarily relies on screening ionizable lipids by  traditional experiments which consumes intensive cost and time. Current study  attempts to apply computational methods to accelerate the LNP development for  mRNA vaccines. Firstly, 325 data samples of mRNA vaccine LNP formulations with  IgG titer were collected. The machine learning algorithm, lightGBM, was used to  build a prediction model with good performance (R2 > 0.87). More importantly, the  critical substructures of ionizable lipids in LNPs were identified by the algorithm,  which well agreed with published results. The animal experimental results showed  that LNP using DLin-MC3-DMA (MC3) as ionizable lipid with an N/P ratio at 6:1  induced higher efficiency in mice than LNP with SM-102, which was consistent with  the model prediction. Molecular dynamic modeling further investigated the molecular  mechanism of LNPs used in the experiment. The result showed that the lipid  molecules aggregated to form LNPs, and mRNA molecules twined around the LNPs.  In summary, the machine learning predictive model for LNP-based mRNA vaccines  was first developed, validated by experiments, and further integrated with molecular  modeling. The prediction model can be used for virtual screening of LNP  formulations in the future.
Efficient Search in Graph Edit Distance: Metric Search Trees vs. Brute Force Verification
Authors: Wenqi Marshall Guo, Jeffrey Uhlmann
Date: 2024-03-15 00:00:00
Tasks: Graph Similarity

This report evaluates the efficiency of Graph Edit Distance (GED) computation for graph similarity search, comparing Cascading Metric Trees (CMT) with brute-force verification. Despite the anticipated advantages of CMT, our findings indicate it does not consistently outperform brute-force methods in speed. The study, based on graph data from PubChem, suggests that the computational complexity of GED-based GSS remains a challenge.
Synthetic Trajectory Generation Through Convolutional Neural Networks
Authors: Jesse Merhi, Erik Buchholz, Salil S. Kanhere
Date: 2024-07-24 00:00:00
Tasks: Image Generation, Privacy Preserving

Location trajectories provide valuable insights for applications from urban planning to pandemic control. However, mobility data can also reveal sensitive information about individuals, such as political opinions, religious beliefs, or sexual orientations. Existing privacy-preserving approaches for publishing this data face a significant utility-privacy trade-off. Releasing synthetic trajectory data generated through deep learning offers a promising solution. Due to the trajectories' sequential nature, most existing models are based on recurrent neural networks (RNNs). However, research in generative adversarial networks (GANs) largely employs convolutional neural networks (CNNs) for image generation. This discrepancy raises the question of whether advances in computer vision can be applied to trajectory generation. In this work, we introduce a Reversible Trajectory-to-CNN Transformation (RTCT) that adapts trajectories into a format suitable for CNN-based models. We integrated this transformation with the well-known DCGAN in a proof-of-concept (PoC) and evaluated its performance against an RNN-based trajectory GAN using four metrics across two datasets. The PoC was superior in capturing spatial distributions compared to the RNN model but had difficulty replicating sequential and temporal properties. Although the PoC's utility is not sufficient for practical applications, the results demonstrate the transformation's potential to facilitate the use of CNNs for trajectory generation, opening up avenues for future research. To support continued research, all source code has been made available under an open-source license.
An Efficient Procedure for Computing Bayesian Network Structure Learning
Authors: Hongming Huang, Joe Suzuki
Date: 2024-07-24 00:00:00
Tasks: Computational Efficiency

We propose a globally optimal Bayesian network structure discovery algorithm based on a progressively leveled scoring approach. Bayesian network structure discovery is a fundamental yet NP-hard problem in the field of probabilistic graphical models, and as the number of variables increases, memory usage grows exponentially. The simple and effective method proposed by Silander and Myllym\"aki has been widely applied in this field, as it incrementally calculates local scores to achieve global optimality. However, existing methods that utilize disk storage, while capable of handling networks with a larger number of variables, introduce issues such as latency, fragmentation, and additional overhead associated with disk I/O operations. To avoid these problems, we explore how to further enhance computational efficiency and reduce peak memory usage using only memory. We introduce an efficient hierarchical computation method that requires only a single traversal of all local structures, retaining only the data and information necessary for the current computation, thereby improving efficiency and significantly reducing memory requirements. Experimental results indicate that our method, when using only memory, not only reduces peak memory usage but also improves computational efficiency compared to existing methods, demonstrating good scalability for handling larger networks and exhibiting stable experimental results. Ultimately, we successfully achieved the processing of a Bayesian network with 28 variables using only memory.
Uncertainty-Based Ensemble Learning For Speech Classification
Authors: Bagus Tris Atmaja, Felix Burkhardt
Date: 2024-07-24 00:00:00
Tasks: Classification, Ensemble Learning

Speech classification has attracted increasing attention due to its wide applications, particularly in classifying physical and mental states. However, these tasks are challenging due to the high variability in speech signals. Ensemble learning has shown promising results when multiple classifiers are combined to improve performance. With recent advancements in hardware development, combining several models is not a limitation in deep learning research and applications. In this paper, we propose an uncertainty-based ensemble learning approach for speech classification. Specifically, we train a set of base features on the same classifier and quantify the uncertainty of their predictions. The predictions are combined using variants of uncertainty calculation to produce the final prediction. The visualization of the effect of uncertainty and its ensemble learning results show potential improvements in speech classification tasks. The proposed method outperforms single models and conventional ensemble learning methods in terms of unweighted accuracy or weighted accuracy.