pwc-archive/papers-with-abstracts huggingface dataset:
Integrated Approach of Gearbox Fault Diagnosis
Authors: Vikash Kumar, Subrata Mukherjee, Somnath Sarangi
Date: 2023-08-27 00:00:00
Tasks: Fault Diagnosis

Gearbox fault diagnosis is one of the most important parts in any industrial systems. Failure of components inside gearbox can lead to a catastrophic failure, uneven breakdown, and financial losses in industrial organization. In that case intelligent maintenance of the gearbox comes into context. This paper presents an integrated gearbox fault diagnosis approach which can easily deploy in online condition monitoring. This work introduces a nonparametric data preprocessing technique i.e., calculus enhanced energy operator (CEEO) to preserve the characteristics frequencies in the noisy and inferred vibrational signal. A set of time domain and spectral domain features are calculated from the raw and CEEO vibration signal and inputted to the multiclass support vector machine (MCSVM) to diagnose the faults on the system. An effective comparison between raw signal and CEEO signal are presented to show the impact of CEEO in gearbox fault diagnosis. The obtained results of this work look very promising and can be implemented in any type of industrial system due to its nonparametric nature.
Learning end-to-end inversion of circular Radon transforms in the partial radial setup
Authors: Deep Ray, Souvik Roy
Date: 2023-08-27 00:00:00

We present a deep learning-based computational algorithm for inversion of circular Radon transforms in the partial radial setup, arising in photoacoustic tomography. We first demonstrate that the truncated singular value decomposition-based method, which is the only traditional algorithm available to solve this problem, leads to severe artifacts which renders the reconstructed field as unusable. With the objective of overcoming this computational bottleneck, we train a ResBlock based U-Net to recover the inferred field that directly operates on the measured data. Numerical results with augmented Shepp-Logan phantoms, in the presence of noisy full and limited view data, demonstrate the superiority of the proposed algorithm.
Integrated Variational Fourier Features for Fast Spatial Modelling with Gaussian Processes
Authors: Talay M Cheema, Carl Edward Rasmussen
Date: 2023-08-27 00:00:00
Tasks: Gaussian Processes

Sparse variational approximations are popular methods for scaling up inference and learning in Gaussian processes to larger datasets. For $N$ training points, exact inference has $O(N^3)$ cost; with $M \ll N$ features, state of the art sparse variational methods have $O(NM^2)$ cost. Recently, methods have been proposed using more sophisticated features; these promise $O(M^3)$ cost, with good performance in low dimensional tasks such as spatial modelling, but they only work with a very limited class of kernels, excluding some of the most commonly used. In this work, we propose integrated Fourier features, which extends these performance benefits to a very broad class of stationary covariance functions. We motivate the method and choice of parameters from a convergence analysis and empirical exploration, and show practical speedup in synthetic and real world spatial regression tasks.
SPEED: Streaming Partition and Parallel Acceleration for Temporal Interaction Graph Embedding
Authors: Xi Chen, Yongxiang Liao, Yun Xiong, Yao Zhang, Siwei Zhang, Jiawei Zhang, Yiheng Sun
Date: 2023-08-27 00:00:00
Tasks: CPU, GPU, Graph Embedding

Temporal Interaction Graphs (TIGs) are widely employed to model intricate real-world systems such as financial systems and social networks. To capture the dynamism and interdependencies of nodes, existing TIG embedding models need to process edges sequentially and chronologically. However, this requirement prevents it from being processed in parallel and struggle to accommodate burgeoning data volumes to GPU. Consequently, many large-scale temporal interaction graphs are confined to CPU processing. Furthermore, a generalized GPU scaling and acceleration approach remains unavailable. To facilitate large-scale TIGs' implementation on GPUs for acceleration, we introduce a novel training approach namely Streaming Edge Partitioning and Parallel Acceleration for Temporal Interaction Graph Embedding (SPEED). The SPEED is comprised of a Streaming Edge Partitioning Component (SEP) which addresses space overhead issue by assigning fewer nodes to each GPU, and a Parallel Acceleration Component (PAC) which enables simultaneous training of different sub-graphs, addressing time overhead issue. Our method can achieve a good balance in computing resources, computing time, and downstream task performance. Empirical validation across 7 real-world datasets demonstrates the potential to expedite training speeds by a factor of up to 19.29x. Simultaneously, resource consumption of a single-GPU can be diminished by up to 69%, thus enabling the multiple GPU-based training and acceleration encompassing millions of nodes and billions of edges. Furthermore, our approach also maintains its competitiveness in downstream tasks.
Hybrid Transformer-RNN Architecture for Household Occupancy Detection Using Low-Resolution Smart Meter Data
Authors: Xinyu Liang, Hao Wang
Date: 2023-08-27 00:00:00
Tasks: energy management, Management

Residential occupancy detection has become an enabling technology in today's urbanized world for various smart home applications, such as building automation, energy management, and improved security and comfort. Digitalization of the energy system provides smart meter data that can be used for occupancy detection in a non-intrusive manner without causing concerns regarding privacy and data security. In particular, deep learning techniques make it possible to infer occupancy from low-resolution smart meter data, such that the need for accurate occupancy detection with privacy preservation can be achieved. Our work is thus motivated to develop a privacy-aware and effective model for residential occupancy detection in contemporary living environments. Our model aims to leverage the advantages of both recurrent neural networks (RNNs), which are adept at capturing local temporal dependencies, and transformers, which are effective at handling global temporal dependencies. Our designed hybrid transformer-RNN model detects residential occupancy using hourly smart meter data, achieving an accuracy of nearly 92\% across households with diverse profiles. We validate the effectiveness of our method using a publicly accessible dataset and demonstrate its performance by comparing it with state-of-the-art models, including attention-based occupancy detection methods.
Towards Generalizable Neural Solvers for Vehicle Routing Problems via Ensemble with Transferrable Local Policy
Authors: Chengrui Gao, Haopu Shang, Ke Xue, Dong Li, Chao Qian
Date: 2023-08-27 00:00:00
Tasks: Combinatorial Optimization

Machine learning has been adapted to help solve NP-hard combinatorial optimization problems. One prevalent way is learning to construct solutions by deep neural networks, which has been receiving more and more attention due to the high efficiency and less requirement for expert knowledge. However, many neural construction methods for Vehicle Routing Problems~(VRPs) focus on synthetic problem instances with specified node distributions and limited scales, leading to poor performance on real-world problems which usually involve complex and unknown node distributions together with large scales. To make neural VRP solvers more practical, we design an auxiliary policy that learns from the local transferable topological features, named local policy, and integrate it with a typical construction policy (which learns from the global information of VRP instances) to form an ensemble policy. With joint training, the aggregated policies perform cooperatively and complementarily to boost generalization. The experimental results on two well-known benchmarks, TSPLIB and CVRPLIB, of travelling salesman problem and capacitated VRP show that the ensemble policy significantly improves both cross-distribution and cross-scale generalization performance, and even performs well on real-world problems with several thousand nodes.
Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective
Authors: Davide Ghio, Yatin Dandi, Florent Krzakala, Lenka Zdeborová
Date: 2023-08-27 00:00:00
Tasks: Denoising

Recent years witnessed the development of powerful generative models based on flows, diffusion or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas. A theoretical analysis of the performance and understanding of the limitations of these methods remain, however, challenging. In this paper, we undertake a step in this direction by analysing the efficiency of sampling by these methods on a class of problems with a known probability distribution and comparing it with the sampling performance of more traditional methods such as the Monte Carlo Markov chain and Langevin dynamics. We focus on a class of probability distribution widely studied in the statistical physics of disordered systems that relate to spin glasses, statistical inference and constraint satisfaction problems. We leverage the fact that sampling via flow-based, diffusion-based or autoregressive networks methods can be equivalently mapped to the analysis of a Bayes optimal denoising of a modified probability measure. Our findings demonstrate that these methods encounter difficulties in sampling stemming from the presence of a first-order phase transition along the algorithm's denoising path. Our conclusions go both ways: we identify regions of parameters where these methods are unable to sample efficiently, while that is possible using standard Monte Carlo or Langevin approaches. We also identify regions where the opposite happens: standard approaches are inefficient while the discussed generative methods work well.
A Bayesian Non-parametric Approach to Generative Models: Integrating Variational Autoencoder and Generative Adversarial Networks using Wasserstein and Maximum Mean Discrepancy
Authors: Forough Fazeli-Asl, Michael Minyi Zhang
Date: 2023-08-27 00:00:00
Tasks: Anomaly Detection, Data Augmentation

We propose a novel generative model within the Bayesian non-parametric learning (BNPL) framework to address some notable failure modes in generative adversarial networks (GANs) and variational autoencoders (VAEs)--these being overfitting in the GAN case and noisy samples in the VAE case. We will demonstrate that the BNPL framework enhances training stability and provides robustness and accuracy guarantees when incorporating the Wasserstein distance and maximum mean discrepancy measure (WMMD) into our model's loss function. Moreover, we introduce a so-called ``triple model'' that combines the GAN, the VAE, and further incorporates a code-GAN (CGAN) to explore the latent space of the VAE. This triple model design generates high-quality, diverse samples, while the BNPL framework, leveraging the WMMD loss function, enhances training stability. Together, these components enable our model to achieve superior performance across various generative tasks. These claims are supported by both theoretical analyses and empirical validation on a wide variety of datasets.
A Comparison of Neural Networks for Wireless Channel Prediction
Authors: Oscar Stenhammar, Gabor Fodor, Carlo Fischione
Date: 2023-08-27 00:00:00
Tasks: Prediction

The performance of modern wireless communications systems depends critically on the quality of the available channel state information (CSI) at the transmitter and receiver. Several previous works have proposed concepts and algorithms that help maintain high quality CSI even in the presence of high mobility and channel aging, such as temporal prediction schemes that employ neural networks. However, it is still unclear which neural network-based scheme provides the best performance in terms of prediction quality, training complexity and practical feasibility. To investigate such a question, this paper first provides an overview of state-of-the-art neural networks applicable to channel prediction and compares their performance in terms of prediction quality. Next, a new comparative analysis is proposed for four promising neural networks with different prediction horizons. The well-known tapped delay channel model recommended by the Third Generation Partnership Program is used for a standardized comparison among the neural networks. Based on this comparative evaluation, the advantages and disadvantages of each neural network are discussed and guidelines for selecting the best-suited neural network in channel prediction applications are given.
Online GentleAdaBoost -- Technical Report
Authors: Chapman Siu
Date: 2023-08-27 00:00:00

We study the online variant of GentleAdaboost, where we combine a weak learner to a strong learner in an online fashion. We provide an approach to extend the batch approach to an online approach with theoretical justifications through application of line search. Finally we compare our online boosting approach with other online approaches across a variety of benchmark datasets.
Improve in-situ life prediction and classification performance by capturing both the present state and evolution rate of battery aging
Authors: Mingyuan Zhao, Yongzhi Zhang
Date: 2023-08-27 00:00:00
Tasks: Gaussian Processes, Prediction

This study develops a methodology by capturing both the battery aging state and degradation rate for improved life prediction performance. The aging state is indicated by six physical features of an equivalent circuit model that are extracted from the voltage relaxation data. And the degradation rate is captured by two features extracted from the differences between the voltage relaxation curves within a moving window (for life prediction), or the differences between the capacity vs. voltage curves at different cycles (for life classification). Two machine learning models, which are constructed based on Gaussian Processes, are used to describe the relationships between these physical features and battery lifetimes for the life prediction and classification, respectively. The methodology is validated with the aging data of 74 battery cells of three different types. Experimental results show that based on only 3-12 minutes' sampling data, the method with novel features predicts accurate battery lifetimes, with the prediction accuracy improved by up to 67.09% compared with the benchmark method. And the batteries are classified into three groups (long, medium, and short) with an overall accuracy larger than 90% based on only two adjacent cycles' information, enabling the highly efficient regrouping of retired batteries.
Interpolation of mountain weather forecasts by machine learning
Authors: Kazuma Iwase, Tomoyuki Takenawa
Date: 2023-08-27 00:00:00
Tasks: Feature Engineering

Recent advances in numerical simulation methods based on physical models and their combination with machine learning have improved the accuracy of weather forecasts. However, the accuracy decreases in complex terrains such as mountainous regions because these methods usually use grids of several kilometers square and simple machine learning models. While deep learning has also made significant progress in recent years, its direct application is difficult to utilize the physical knowledge used in the simulation. This paper proposes a method that uses machine learning to interpolate future weather in mountainous regions using forecast data from surrounding plains and past observed data to improve weather forecasts in mountainous regions. We focus on mountainous regions in Japan and predict temperature and precipitation mainly using LightGBM as a machine learning model. Despite the use of a small dataset, through feature engineering and model tuning, our method partially achieves improvements in the RMSE with significantly less training time.
Universal Graph Continual Learning
Authors: Thanh Duc Hoang, Do Viet Tung, Duy-Hung Nguyen, Bao-Sinh Nguyen, Huy Hoang Nguyen, Hung Le
Date: 2023-08-27 00:00:00
Tasks: Continual Learning, Graph Classification, Graph Learning, Node Classification

We address catastrophic forgetting issues in graph learning as incoming data transits from one to another graph distribution. Whereas prior studies primarily tackle one setting of graph continual learning such as incremental node classification, we focus on a universal approach wherein each data point in a task can be a node or a graph, and the task varies from node to graph classification. We propose a novel method that enables graph neural networks to excel in this universal setting. Our approach perseveres knowledge about past tasks through a rehearsal mechanism that maintains local and global structure consistency across the graphs. We benchmark our method against various continual learning baselines in real-world graph datasets and achieve significant improvement in average performance and forgetting across tasks.
Multivariate time series classification with dual attention network
Authors: Mojtaba A. Farahani, Tara Eslaminokandeh
Date: 2023-08-26 00:00:00
Tasks: Classification, Time Series, Time Series Classification

One of the topics in machine learning that is becoming more and more relevant is multivariate time series classification. Current techniques concentrate on identifying the local important sequence segments or establishing the global long-range dependencies. They frequently disregard the merged data from both global and local features, though. Using dual attention, we explore a novel network (DA-Net) in this research to extract local and global features for multivariate time series classification. The two distinct layers that make up DA-Net are the Squeeze-Excitation Window Attention (SEWA) layer and the Sparse Self-Attention within Windows (SSAW) layer. DA- Net can mine essential local sequence fragments that are necessary for establishing global long-range dependencies based on the two expanded layers.
Deep learning assisted robust detection techniques for a chipless RFID sensor tag
Authors: Nadeem Rather, Roy B. V. B. Simorangkir, John L. Buckley, Brendan O'Flynn, Salvatore Tedesco
Date: 2023-08-26 00:00:00
Tasks: TAG

In this paper, we present a new approach for robust reading of identification and sensor data from chipless RFID sensor tags. For the first time, Machine Learning (ML) and Deep Learning (DL) regression modelling techniques are applied to a dataset of measured Radar Cross Section (RCS) data that has been derived from large-scale robotic measurements of custom-designed, 3-bit chipless RFID sensor tags. The robotic system is implemented using the first-of-its-kind automated data acquisition method using an ur16e industry-standard robot. A large data set of 9,600 Electromagnetic (EM) RCS signatures collected using the automated system is used to train and validate four ML models and four 1-dimensional Convolutional Neural Network (1D CNN) architectures. For the first time, we report an end-to-end design and implementation methodology for robust detection of identification (ID) and sensing data using ML/DL models. Also, we report, for the first time, the effect of varying tag surface shapes, tilt angles, and read ranges that were incorporated into the training of models for robust detection of ID and sensing values. The results show that all the models were able to generalise well on the given data. However, the 1D CNN models outperformed the conventional ML models in the detection of ID and sensing values. The best 1D CNN model architectures performed well with a low Root Mean Square Error (RSME) of 0.061 (0.87%) for tag ID and 0.0241 (3.44%) error for the capacitive sensing.
A small vocabulary database of ultrasound image sequences of vocal tract dynamics
Authors: Margareth Castillo, Felipe Rubio, Dagoberto Porras, Sonia H. Contreras-Ortiz, Alexander Sepúlveda
Date: 2023-08-26 00:00:00

This paper presents a new database consisting of concurrent articulatory and acoustic speech data. The articulatory data correspond to ultrasound videos of the vocal tract dynamics, which allow the visualization of the tongue upper contour during the speech production process. Acoustic data is composed of 30 short sentences that were acquired by a directional cardioid microphone. This database includes data from 17 young subjects (8 male and 9 female) from the Santander region in Colombia, who reported not having any speech pathology.
A transport approach to sequential simulation-based inference
Authors: Paul-Baptiste Rubio, Youssef Marzouk, Matthew Parno
Date: 2023-08-26 00:00:00
Tasks: Bayesian Inference, parameter estimation, Sequential Bayesian Inference

We present a new transport-based approach to efficiently perform sequential Bayesian inference of static model parameters. The strategy is based on the extraction of conditional distribution from the joint distribution of parameters and data, via the estimation of structured (e.g., block triangular) transport maps. This gives explicit surrogate models for the likelihood functions and their gradients. This allow gradient-based characterizations of posterior density via transport maps in a model-free, online phase. This framework is well suited for parameter estimation in case of complex noise models including nuisance parameters and when the forward model is only known as a black box. The numerical application of this method is performed in the context of characterization of ice thickness with conductivity measurements.
A Two-Dimensional Deep Network for RF-based Drone Detection and Identification Towards Secure Coverage Extension
Authors: Zixiao Zhao, Qinghe Du, Xiang Yao, Lei Lu, Shijiao Zhang
Date: 2023-08-26 00:00:00

As drones become increasingly prevalent in human life, they also raises security concerns such as unauthorized access and control, as well as collisions and interference with manned aircraft. Therefore, ensuring the ability to accurately detect and identify between different drones holds significant implications for coverage extension. Assisted by machine learning, radio frequency (RF) detection can recognize the type and flight mode of drones based on the sampled drone signals. In this paper, we first utilize Short-Time Fourier. Transform (STFT) to extract two-dimensional features from the raw signals, which contain both time-domain and frequency-domain information. Then, we employ a Convolutional Neural Network (CNN) built with ResNet structure to achieve multi-class classifications. Our experimental results show that the proposed ResNet-STFT can achieve higher accuracy and faster convergence on the extended dataset. Additionally, it exhibits balanced performance compared to other baselines on the raw dataset.