pwc-archive/papers-with-abstracts huggingface dataset:
Predicting building types and functions at transnational scale
Authors: Jonas Fill, Michael Eichelbeck, Michael Ebner
Date: 2024-09-15 00:00:00
Tasks: Graph Neural Network

Building-specific knowledge such as building type and function information is important for numerous energy applications. However, comprehensive datasets containing this information for individual households are missing in many regions of Europe. For the first time, we investigate whether it is feasible to predict building types and functional classes at a European scale based on only open GIS datasets available across countries. We train a graph neural network (GNN) classifier on a large-scale graph dataset consisting of OpenStreetMap (OSM) buildings across the EU, Norway, Switzerland, and the UK. To efficiently perform training using the large-scale graph, we utilize localized subgraphs. A graph transformer model achieves a high Cohen's kappa coefficient of 0.754 when classifying buildings into 9 classes, and a very high Cohen's kappa coefficient of 0.844 when classifying buildings into the residential and non-residential classes. The experimental results imply three core novel contributions to literature. Firstly, we show that building classification across multiple countries is possible using a multi-source dataset consisting of information about 2D building shape, land use, degree of urbanization, and countries as input, and OSM tags as ground truth. Secondly, our results indicate that GNN models that consider contextual information about building neighborhoods improve predictive performance compared to models that only consider individual buildings and ignore the neighborhood. Thirdly, we show that training with GNNs on localized subgraphs instead of standard GNNs improves performance for the task of building classification.
Extrapolative ML Models for Copolymers
Authors: Israrul H. Hashmi, Himanshu, Rahul Karmakar, Tarak K Patra
Date: 2024-09-15 00:00:00

Machine learning models have been progressively used for predicting materials properties. These models can be built using pre-existing data and are useful for rapidly screening the physicochemical space of a material, which is astronomically large. However, ML models are inherently interpolative, and their efficacy for searching candidates outside a material's known range of property is unresolved. Moreover, the performance of an ML model is intricately connected to its learning strategy and the volume of training data. Here, we determine the relationship between the extrapolation ability of an ML model, the size and range of its training dataset, and its learning approach. We focus on a canonical problem of predicting the properties of a copolymer as a function of the sequence of its monomers. Tree search algorithms, which learn the similarity between polymer structures, are found to be inefficient for extrapolation. Conversely, the extrapolation capability of neural networks and XGBoost models, which attempt to learn the underlying functional correlation between the structure and property of polymers, show strong correlations with the volume and range of training data. These findings have important implications on ML-based new material development.
Mitigating Dimensionality in 2D Rectangle Packing Problem under Reinforcement Learning Schema
Authors: Waldemar Kołodziejczyk, Mariusz Kaleta
Date: 2024-09-15 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper explores the application of Reinforcement Learning (RL) to the two-dimensional rectangular packing problem. We propose a reduced representation of the state and action spaces that allow us for high granularity. Leveraging UNet architecture and Proximal Policy Optimization (PPO), we achieved a model that is comparable to the MaxRect heuristic. However, our approach has great potential to be generalized to nonrectangular packing problems and complex constraints.
Model Selection Through Model Sorting
Authors: Mohammad Ali Hajiani, Babak Seyfe
Date: 2024-09-15 00:00:00
Tasks: model, Model Selection, NER

We propose a novel approach to select the best model of the data. Based on the exclusive properties of the nested models, we find the most parsimonious model containing the risk minimizer predictor. We prove the existence of probable approximately correct (PAC) bounds on the difference of the minimum empirical risk of two successive nested models, called successive empirical excess risk (SEER). Based on these bounds, we propose a model order selection method called nested empirical risk (NER). By the sorted NER (S-NER) method to sort the models intelligently, the minimum risk decreases. We construct a test that predicts whether expanding the model decreases the minimum risk or not. With a high probability, the NER and S-NER choose the true model order and the most parsimonious model containing the risk minimizer predictor, respectively. We use S-NER model selection in the linear regression and show that, the S-NER method without any prior information can outperform the accuracy of feature sorting algorithms like orthogonal matching pursuit (OMP) that aided with prior knowledge of the true model order. Also, in the UCR data set, the NER method reduces the complexity of the classification of UCR datasets dramatically, with a negligible loss of accuracy.
Conditional sampling within generative diffusion models
Authors: Zheng Zhao, Ziwei Luo, Jens Sjölund, Thomas B. Schön
Date: 2024-09-15 00:00:00

Generative diffusions are a powerful class of Monte Carlo samplers that leverage bridging Markov processes to approximate complex, high-dimensional distributions, such as those found in image processing and language models. Despite their success in these domains, an important open challenge remains: extending these techniques to sample from conditional distributions, as required in, for example, Bayesian inverse problems. In this paper, we present a comprehensive review of existing computational approaches to conditional sampling within generative diffusion models. Specifically, we highlight key methodologies that either utilise the joint distribution, or rely on (pre-trained) marginal distributions with explicit likelihoods, to construct conditional generative samplers.
Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancement
Authors: Yudong Yang, Zhan Liu, Wenyi Yu, Guangzhi Sun, Qiuqiang Kong, Chao Zhang
Date: 2024-09-15 00:00:00

Diffusion-based generative models have recently achieved remarkable results in speech and vocal enhancement due to their ability to model complex speech data distributions. While these models generalize well to unseen acoustic environments, they may not achieve the same level of fidelity as the discriminative models specifically trained to enhance particular acoustic conditions. In this paper, we propose Ex-Diff, a novel score-based diffusion model that integrates the latent representations produced by a discriminative model to improve speech and vocal enhancement, which combines the strengths of both generative and discriminative models. Experimental results on the widely used MUSDB dataset show relative improvements of 3.7% in SI-SDR and 10.0% in SI-SIR compared to the baseline diffusion model for speech and vocal enhancement tasks, respectively. Additionally, case studies are provided to further illustrate and analyze the complementary nature of generative and discriminative models in this context.
HJ-sampler: A Bayesian sampler for inverse problems of a stochastic process by leveraging Hamilton-Jacobi PDEs and score-based generative models
Authors: Tingwei Meng, Zongren Zou, Jérôme Darbon, George Em Karniadakis
Date: 2024-09-15 00:00:00
Tasks: Bayesian Inference

The interplay between stochastic processes and optimal control has been extensively explored in the literature. With the recent surge in the use of diffusion models, stochastic processes have increasingly been applied to sample generation. This paper builds on the log transform, known as the Cole-Hopf transform in Brownian motion contexts, and extends it within a more abstract framework that includes a linear operator. Within this framework, we found that the well-known relationship between the Cole-Hopf transform and optimal transport is a particular instance where the linear operator acts as the infinitesimal generator of a stochastic process. We also introduce a novel scenario where the linear operator is the adjoint of the generator, linking to Bayesian inference under specific initial and terminal conditions. Leveraging this theoretical foundation, we develop a new algorithm, named the HJ-sampler, for Bayesian inference for the inverse problem of a stochastic differential equation with given terminal observations. The HJ-sampler involves two stages: (1) solving the viscous Hamilton-Jacobi partial differential equations, and (2) sampling from the associated stochastic optimal control problem. Our proposed algorithm naturally allows for flexibility in selecting the numerical solver for viscous HJ PDEs. We introduce two variants of the solver: the Riccati-HJ-sampler, based on the Riccati method, and the SGM-HJ-sampler, which utilizes diffusion models. We demonstrate the effectiveness and flexibility of the proposed methods by applying them to solve Bayesian inverse problems involving various stochastic processes and prior distributions, including applications that address model misspecifications and quantifying model uncertainty.
Machine learning assisted screening of metal binary alloys for anode materials
Authors: Xingyue Shi, Linming Zhou, YuHui Huang, Yongjun Wu, Zijian Hong
Date: 2024-09-15 00:00:00

In the dynamic and rapidly advancing battery field, alloy anode materials are a focal point due to their superior electrochemical performance. Traditional screening methods are inefficient and time-consuming. Our research introduces a machine learning-assisted strategy to expedite the discovery and optimization of these materials. We compiled a vast dataset from the MP and AFLOW databases, encompassing tens of thousands of alloy compositions and properties. Utilizing a CGCNN, we accurately predicted the potential and specific capacity of alloy anodes, validated against experimental data. This approach identified approximately 120 low potential and high specific capacity alloy anodes suitable for various battery systems including Li, Na, K, Zn, Mg, Ca, and Al-based. Our method not only streamlines the screening of battery anode materials but also propels the advancement of battery material research and innovation in energy storage technology.
A Statistical Viewpoint on Differential Privacy: Hypothesis Testing, Representation and Blackwell's Theorem
Authors: Weijie J. Su
Date: 2024-09-14 00:00:00
Tasks: Informativeness, Privacy Preserving

Differential privacy is widely considered the formal privacy for privacy-preserving data analysis due to its robust and rigorous guarantees, with increasingly broad adoption in public services, academia, and industry. Despite originating in the cryptographic context, in this review paper we argue that, fundamentally, differential privacy can be considered a \textit{pure} statistical concept. By leveraging David Blackwell's informativeness theorem, our focus is to demonstrate based on prior work that all definitions of differential privacy can be formally motivated from a hypothesis testing perspective, thereby showing that hypothesis testing is not merely convenient but also the right language for reasoning about differential privacy. This insight leads to the definition of $f$-differential privacy, which extends other differential privacy definitions through a representation theorem. We review techniques that render $f$-differential privacy a unified framework for analyzing privacy bounds in data analysis and machine learning. Applications of this differential privacy definition to private deep learning, private convex optimization, shuffled mechanisms, and U.S.\ Census data are discussed to highlight the benefits of analyzing privacy bounds under this framework compared to existing alternatives.
Multi-Microphone and Multi-Modal Emotion Recognition in Reverberant Environment
Authors: Ohad Cohen, Gershon Hazan, Sharon Gannot
Date: 2024-09-14 00:00:00
Tasks: Emotion Recognition

This paper presents a Multi-modal Emotion Recognition (MER) system designed to enhance emotion recognition accuracy in challenging acoustic conditions. Our approach combines a modified and extended Hierarchical Token-semantic Audio Transformer (HTS-AT) for multi-channel audio processing with an R(2+1)D Convolutional Neural Networks (CNN) model for video analysis. We evaluate our proposed method on a reverberated version of the Ryerson audio-visual database of emotional speech and song (RAVDESS) dataset using synthetic and real-world Room Impulse Responsess (RIRs). Our results demonstrate that integrating audio and video modalities yields superior performance compared to uni-modal approaches, especially in challenging acoustic conditions. Moreover, we show that the multimodal (audiovisual) approach that utilizes multiple microphones outperforms its single-microphone counterpart.
Deep Fast Machine Learning Utils: A Python Library for Streamlined Machine Learning Prototyping
Authors: Fabi Prezja
Date: 2024-09-14 00:00:00
Tasks: feature selection, Management

Machine learning (ML) research and application often involve time-consuming steps such as model architecture prototyping, feature selection, and dataset preparation. To support these tasks, we introduce the Deep Fast Machine Learning Utils (DFMLU) library, which provides tools designed to automate and enhance aspects of these processes. Compatible with frameworks like TensorFlow, Keras, and Scikit-learn, DFMLU offers functionalities that support model development and data handling. The library includes methods for dense neural network search, advanced feature selection, and utilities for data management and visualization of training outcomes. This manuscript presents an overview of DFMLU's functionalities, providing Python examples for each tool.
Evaluating probabilistic and data-driven inference models for fiber-coupled NV-diamond temperature sensors
Authors: Shraddha Rajpal, Zeeshan Ahmed, Tyrus Berry
Date: 2024-09-14 00:00:00

We evaluate the impact of inference model on uncertainties when using continuous wave Optically Detected Magnetic Resonance (ODMR) measurements to infer temperature. Our approach leverages a probabilistic feedforward inference model designed to maximize the likelihood of observed ODMR spectra through automatic differentiation. This model effectively utilizes the temperature dependence of spin Hamiltonian parameters to infer temperature from spectral features in the ODMR data. We achieve prediction uncertainty of $\pm$ 1 K across a temperature range of 243 K to 323 K. To benchmark our probabilistic model, we compare it with a non-parametric peak-finding technique and data-driven methodologies such as Principal Component Regression (PCR) and a 1D Convolutional Neural Network (CNN). We find that when validated against out-of-sample dataset that encompasses the same temperature range as the training dataset, data driven methods can show uncertainties that are as much as 0.67 K lower without incorporating expert-level understanding of the spectroscopic-temperature relationship. However, our results show that the probabilistic model outperforms both PCR and CNN when tasked with extrapolating beyond the temperature range used in training set, indicating robustness and generalizability. In contrast, data-driven methods like PCR and CNN demonstrate up to ten times worse uncertainties when tasked with extrapolating outside their training data range.
Neumann Series-based Neural Operator for Solving Inverse Medium Problem
Authors: Ziyang Liu, Fukai Chen, Junqing Chen, Lingyun Qiu, Zuoqiang Shi
Date: 2024-09-14 00:00:00

The inverse medium problem, inherently ill-posed and nonlinear, presents significant computational challenges. This study introduces a novel approach by integrating a Neumann series structure within a neural network framework to effectively handle multiparameter inputs. Experiments demonstrate that our methodology not only accelerates computations but also significantly enhances generalization performance, even with varying scattering properties and noisy data. The robustness and adaptability of our framework provide crucial insights and methodologies, extending its applicability to a broad spectrum of scattering problems. These advancements mark a significant step forward in the field, offering a scalable solution to traditionally complex inverse problems.
MALADY: Multiclass Active Learning with Auction Dynamics on Graphs
Authors: Gokul Bhusal, Kevin Miller, Ekaterina Merkurjev
Date: 2024-09-14 00:00:00
Tasks: Active Learning

Active learning enhances the performance of machine learning methods, particularly in semi-supervised cases, by judiciously selecting a limited number of unlabeled data points for labeling, with the goal of improving the performance of an underlying classifier. In this work, we introduce the Multiclass Active Learning with Auction Dynamics on Graphs (MALADY) framework which leverages the auction dynamics algorithm on similarity graphs for efficient active learning. In particular, we generalize the auction dynamics algorithm on similarity graphs for semi-supervised learning in [24] to incorporate a more general optimization functional. Moreover, we introduce a novel active learning acquisition function that uses the dual variable of the auction algorithm to measure the uncertainty in the classifier to prioritize queries near the decision boundaries between different classes. Lastly, using experiments on classification tasks, we evaluate the performance of our proposed method and show that it exceeds that of comparison algorithms.
Learning to enhance multi-legged robot on rugged landscapes
Authors: Juntao He, Baxi Chong, Zhaochen Xu, Sehoon Ha, Daniel I. Goldman
Date: 2024-09-14 00:00:00
Tasks: MuJoCo

Navigating rugged landscapes poses significant challenges for legged locomotion. Multi-legged robots (those with 6 and greater) offer a promising solution for such terrains, largely due to their inherent high static stability, resulting from a low center of mass and wide base of support. Such systems require minimal effort to maintain balance. Recent studies have shown that a linear controller, which modulates the vertical body undulation of a multi-legged robot in response to shifts in terrain roughness, can ensure reliable mobility on challenging terrains. However, the potential of a learning-based control framework that adjusts multiple parameters to address terrain heterogeneity remains underexplored. We posit that the development of an experimentally validated physics-based simulator for this robot can rapidly advance capabilities by allowing wide parameter space exploration. Here we develop a MuJoCo-based simulator tailored to this robotic platform and use the simulation to develop a reinforcement learning-based control framework that dynamically adjusts horizontal and vertical body undulation, and limb stepping in real-time. Our approach improves robot performance in simulation, laboratory experiments, and outdoor tests. Notably, our real-world experiments reveal that the learning-based controller achieves a 30\% to 50\% increase in speed compared to a linear controller, which only modulates vertical body waves. We hypothesize that the superior performance of the learning-based controller arises from its ability to adjust multiple parameters simultaneously, including limb stepping, horizontal body wave, and vertical body wave.
Hyperedge Representations with Hypergraph Wavelets: Applications to Spatial Transcriptomics
Authors: Xingzhi Sun, Charles Xu, João F. Rocha, Chen Liu, Benjamin Hollander-Bodie, Laney Goldman, Marcello DiStasio, Michael Perlmutter, Smita Krishnaswamy
Date: 2024-09-14 00:00:00

In many data-driven applications, higher-order relationships among multiple objects are essential in capturing complex interactions. Hypergraphs, which generalize graphs by allowing edges to connect any number of nodes, provide a flexible and powerful framework for modeling such higher-order relationships. In this work, we introduce hypergraph diffusion wavelets and describe their favorable spectral and spatial properties. We demonstrate their utility for biomedical discovery in spatially resolved transcriptomics by applying the method to represent disease-relevant cellular niches for Alzheimer's disease.
PIP-Loco: A Proprioceptive Infinite Horizon Planning Framework for Quadrupedal Robot Locomotion
Authors: Aditya Shirwatkar, Naman Saxena, Kishore Chandra, Shishir Kolathaya
Date: 2024-09-14 00:00:00
Tasks: Model Predictive Control, Reinforcement Learning (RL)

A core strength of Model Predictive Control (MPC) for quadrupedal locomotion has been its ability to enforce constraints and provide interpretability of the sequence of commands over the horizon. However, despite being able to plan, MPC struggles to scale with task complexity, often failing to achieve robust behavior on rapidly changing surfaces. On the other hand, model-free Reinforcement Learning (RL) methods have outperformed MPC on multiple terrains, showing emergent motions but inherently lack any ability to handle constraints or perform planning. To address these limitations, we propose a framework that integrates proprioceptive planning with RL, allowing for agile and safe locomotion behaviors through the horizon. Inspired by MPC, we incorporate an internal model that includes a velocity estimator and a Dreamer module. During training, the framework learns an expert policy and an internal model that are co-dependent, facilitating exploration for improved locomotion behaviors. During deployment, the Dreamer module solves an infinite-horizon MPC problem, adapting actions and velocity commands to respect the constraints. We validate the robustness of our training framework through ablation studies on internal model components and demonstrate improved robustness to training noise. Finally, we evaluate our approach across multi-terrain scenarios in both simulation and hardware.
Weather Prediction Using CNN-LSTM for Time Series Analysis: A Case Study on Delhi Temperature Data
Authors: Bangyu Li, Yang Qian
Date: 2024-09-14 00:00:00
Tasks: energy management, Time Series, Time Series Analysis, Time Series Forecasting, Weather Forecasting

As global climate change intensifies, accurate weather forecasting is increasingly crucial for sectors such as agriculture, energy management, and environmental protection. Traditional methods, which rely on physical and statistical models, often struggle with complex, nonlinear, and time-varying data, underscoring the need for more advanced techniques. This study explores a hybrid CNN-LSTM model to enhance temperature forecasting accuracy for the Delhi region, using historical meteorological data from 1996 to 2017. We employed both direct and indirect methods, including comprehensive data preprocessing and exploratory analysis, to construct and train our model. The CNN component effectively extracts spatial features, while the LSTM captures temporal dependencies, leading to improved prediction accuracy. Experimental results indicate that the CNN-LSTM model significantly outperforms traditional forecasting methods in terms of both accuracy and stability, with a mean square error (MSE) of 3.26217 and a root mean square error (RMSE) of 1.80615. The hybrid model demonstrates its potential as a robust tool for temperature prediction, offering valuable insights for meteorological forecasting and related fields. Future research should focus on optimizing model architecture, exploring additional feature extraction techniques, and addressing challenges such as overfitting and computational complexity. This approach not only advances temperature forecasting but also provides a foundation for applying deep learning to other time series forecasting tasks.