pwc-archive/papers-with-abstracts huggingface dataset:
Scalable and Robust Tensor Ring Decomposition for Large-scale Data
Authors: Yicong He, George K. Atia
Date: 2023-05-15 00:00:00

Tensor ring (TR) decomposition has recently received increased attention due to its superior expressive performance for high-order tensors. However, the applicability of traditional TR decomposition algorithms to real-world applications is hindered by prevalent large data sizes, missing entries, and corruption with outliers. In this work, we propose a scalable and robust TR decomposition algorithm capable of handling large-scale tensor data with missing entries and gross corruptions. We first develop a novel auto-weighted steepest descent method that can adaptively fill the missing entries and identify the outliers during the decomposition process. Further, taking advantage of the tensor ring model, we develop a novel fast Gram matrix computation (FGMC) approach and a randomized subtensor sketching (RStS) strategy which yield significant reduction in storage and computational complexity. Experimental results demonstrate that the proposed method outperforms existing TR decomposition methods in the presence of outliers, and runs significantly faster than existing robust tensor completion algorithms.
Adaptive Federated Pruning in Hierarchical Wireless Networks
Authors: Xiaonan Liu, Shiqiang Wang, Yansha Deng, Arumugam Nallanathan
Date: 2023-05-15 00:00:00
Tasks: Federated Learning, Privacy Preserving

Federated Learning (FL) is a promising privacy-preserving distributed learning framework where a server aggregates models updated by multiple devices without accessing their private datasets. Hierarchical FL (HFL), as a device-edge-cloud aggregation hierarchy, can enjoy both the cloud server's access to more datasets and the edge servers' efficient communications with devices. However, the learning latency increases with the HFL network scale due to the increasing number of edge servers and devices with limited local computation capability and communication bandwidth. To address this issue, in this paper, we introduce model pruning for HFL in wireless networks to reduce the neural network scale. We present the convergence analysis of an upper on the l2 norm of gradients for HFL with model pruning, analyze the computation and communication latency of the proposed model pruning scheme, and formulate an optimization problem to maximize the convergence rate under a given latency threshold by jointly optimizing the pruning ratio and wireless resource allocation. By decoupling the optimization problem and using Karush Kuhn Tucker (KKT) conditions, closed-form solutions of pruning ratio and wireless resource allocation are derived. Simulation results show that our proposed HFL with model pruning achieves similar learning accuracy compared with the HFL without model pruning and reduces about 50 percent communication cost.
Algorithmic Censoring in Dynamic Learning Systems
Authors: Jennifer Chien, Margaret Roberts, Berk Ustun
Date: 2023-05-15 00:00:00

Dynamic learning systems subject to selective labeling exhibit censoring, i.e. persistent negative predictions assigned to one or more subgroups of points. In applications like consumer finance, this results in groups of applicants that are persistently denied and thus never enter into the training data. In this work, we formalize censoring, demonstrate how it can arise, and highlight difficulties in detection. We consider safeguards against censoring - recourse and randomized-exploration - both of which ensure we collect labels for points that would otherwise go unobserved. The resulting techniques allow examples from censored groups to enter into the training data and correct the model. Our results highlight the otherwise unmeasured harms of censoring and demonstrate the effectiveness of mitigation strategies across a range of data generating processes.
SKI to go Faster: Accelerating Toeplitz Neural Networks via Asymmetric Kernels
Authors: Alexander Moreno, Jonathan Mei, Luke Walters
Date: 2023-05-15 00:00:00

Toeplitz Neural Networks (TNNs) (Qin et. al. 2023) are a recent sequence model with impressive results. They require O(n log n) computational complexity and O(n) relative positional encoder (RPE) multi-layer perceptron (MLP) and decay bias calls. We aim to reduce both. We first note that the RPE is a non-SPD (symmetric positive definite) kernel and the Toeplitz matrices are pseudo-Gram matrices. Further 1) the learned kernels display spiky behavior near the main diagonals with otherwise smooth behavior; 2) the RPE MLP is slow. For bidirectional models, this motivates a sparse plus low-rank Toeplitz matrix decomposition. For the sparse component's action, we do a small 1D convolution. For the low rank component, we replace the RPE MLP with linear interpolation and use asymmetric Structured Kernel Interpolation (SKI) (Wilson et. al. 2015) for O(n) complexity: we provide rigorous error analysis. For causal models, "fast" causal masking (Katharopoulos et. al. 2020) negates SKI's benefits. Working in the frequency domain, we avoid an explicit decay bias. To enforce causality, we represent the kernel via the real part of its frequency response using the RPE and compute the imaginary part via a Hilbert transform. This maintains O(n log n) complexity but achieves an absolute speedup. Modeling the frequency response directly is also competitive for bidirectional training, using one fewer FFT. We set a speed state of the art on Long Range Arena (Tay et. al. 2020) with minimal score degradation.
DATED: Guidelines for Creating Synthetic Datasets for Engineering Design Applications
Authors: Cyril Picard, Jürg Schiffmann, Faez Ahmed
Date: 2023-05-15 00:00:00
Tasks: Diversity

Exploiting the recent advancements in artificial intelligence, showcased by ChatGPT and DALL-E, in real-world applications necessitates vast, domain-specific, and publicly accessible datasets. Unfortunately, the scarcity of such datasets poses a significant challenge for researchers aiming to apply these breakthroughs in engineering design. Synthetic datasets emerge as a viable alternative. However, practitioners are often uncertain about generating high-quality datasets that accurately represent real-world data and are suitable for the intended downstream applications. This study aims to fill this knowledge gap by proposing comprehensive guidelines for generating, annotating, and validating synthetic datasets. The trade-offs and methods associated with each of these aspects are elaborated upon. Further, the practical implications of these guidelines are illustrated through the creation of a turbo-compressors dataset. The study underscores the importance of thoughtful sampling methods to ensure the appropriate size, diversity, utility, and realism of a dataset. It also highlights that design diversity does not equate to performance diversity or realism. By employing test sets that represent uniform, real, or task-specific samples, the influence of sample size and sampling strategy is scrutinized. Overall, this paper offers valuable insights for researchers intending to create and publish synthetic datasets for engineering design, thereby paving the way for more effective applications of AI advancements in the field. The code and data for the dataset and methods are made publicly accessible at https://github.com/cyrilpic/radcomp .
Gaussian Process Port-Hamiltonian Systems: Bayesian Learning with Physics Prior
Authors: Thomas Beckers, Jacob Seidman, Paris Perdikaris, George J. Pappas
Date: 2023-05-15 00:00:00
Tasks: Uncertainty Quantification

Data-driven approaches achieve remarkable results for the modeling of complex dynamics based on collected data. However, these models often neglect basic physical principles which determine the behavior of any real-world system. This omission is unfavorable in two ways: The models are not as data-efficient as they could be by incorporating physical prior knowledge, and the model itself might not be physically correct. We propose Gaussian Process Port-Hamiltonian systems (GP-PHS) as a physics-informed Bayesian learning approach with uncertainty quantification. The Bayesian nature of GP-PHS uses collected data to form a distribution over all possible Hamiltonians instead of a single point estimate. Due to the underlying physics model, a GP-PHS generates passive systems with respect to designated inputs and outputs. Further, the proposed approach preserves the compositional nature of Port-Hamiltonian systems.
Physics-enhanced Gaussian Process Variational Autoencoder
Authors: Thomas Beckers, Qirui Wu, George J. Pappas
Date: 2023-05-15 00:00:00

Variational autoencoders allow to learn a lower-dimensional latent space based on high-dimensional input/output data. Using video clips as input data, the encoder may be used to describe the movement of an object in the video without ground truth data (unsupervised learning). Even though the object's dynamics is typically based on first principles, this prior knowledge is mostly ignored in the existing literature. Thus, we propose a physics-enhanced variational autoencoder that places a physical-enhanced Gaussian process prior on the latent dynamics to improve the efficiency of the variational autoencoder and to allow physically correct predictions. The physical prior knowledge expressed as linear dynamical system is here reflected by the Green's function and included in the kernel function of the Gaussian process. The benefits of the proposed approach are highlighted in a simulation with an oscillating particle.
Survey of Malware Analysis through Control Flow Graph using Machine Learning
Authors: Shaswata Mitra, Stephen A. Torri, Sudip Mittal
Date: 2023-05-15 00:00:00
Tasks: Malware Analysis, Malware Detection

Malware is a significant threat to the security of computer systems and networks which requires sophisticated techniques to analyze the behavior and functionality for detection. Traditional signature-based malware detection methods have become ineffective in detecting new and unknown malware due to their rapid evolution. One of the most promising techniques that can overcome the limitations of signature-based detection is to use control flow graphs (CFGs). CFGs leverage the structural information of a program to represent the possible paths of execution as a graph, where nodes represent instructions and edges represent control flow dependencies. Machine learning (ML) algorithms are being used to extract these features from CFGs and classify them as malicious or benign. In this survey, we aim to review some state-of-the-art methods for malware detection through CFGs using ML, focusing on the different ways of extracting, representing, and classifying. Specifically, we present a comprehensive overview of different types of CFG features that have been used as well as different ML algorithms that have been applied to CFG-based malware detection. We provide an in-depth analysis of the challenges and limitations of these approaches, as well as suggest potential solutions to address some open problems and promising future directions for research in this field.
Autoencoder-based Anomaly Detection in Streaming Data with Incremental Learning and Concept Drift Adaptation
Authors: Jin Li, Kleanthis Malialis, Marios M. Polycarpou
Date: 2023-05-15 00:00:00
Tasks: Anomaly Detection, Drift Detection, Incremental Learning

In our digital universe nowadays, enormous amount of data are produced in a streaming manner in a variety of application areas. These data are often unlabelled. In this case, identifying infrequent events, such as anomalies, poses a great challenge. This problem becomes even more difficult in non-stationary environments, which can cause deterioration of the predictive performance of a model. To address the above challenges, the paper proposes an autoencoder-based incremental learning method with drift detection (strAEm++DD). Our proposed method strAEm++DD leverages on the advantages of both incremental learning and drift detection. We conduct an experimental study using real-world and synthetic datasets with severe or extreme class imbalance, and provide an empirical analysis of strAEm++DD. We further conduct a comparative study, showing that the proposed method significantly outperforms existing baseline and advanced methods.
One Forward is Enough for Neural Network Training via Likelihood Ratio Method
Authors: Jinyang Jiang, Zeliang Zhang, Chenliang Xu, Zhaofei Yu, Yijie Peng
Date: 2023-05-15 00:00:00

While backpropagation (BP) is the mainstream approach for gradient computation in neural network training, its heavy reliance on the chain rule of differentiation constrains the designing flexibility of network architecture and training pipelines. We avoid the recursive computation in BP and develop a unified likelihood ratio (ULR) method for gradient estimation with just one forward propagation. Not only can ULR be extended to train a wide variety of neural network architectures, but the computation flow in BP can also be rearranged by ULR for better device adaptation. Moreover, we propose several variance reduction techniques to further accelerate the training process. Our experiments offer numerical results across diverse aspects, including various neural network training scenarios, computation flow rearrangement, and fine-tuning of pre-trained models. All findings demonstrate that ULR effectively enhances the flexibility of neural network training by permitting localized module training without compromising the global objective and significantly boosts the network robustness.
Causal Analysis for Robust Interpretability of Neural Networks
Authors: Ola Ahmad, Nicolas Bereux, Loïc Baret, Vahid Hashemi, Freddy Lecue
Date: 2023-05-15 00:00:00
Tasks: Attribute, image-classification, Image Classification

Interpreting the inner function of neural networks is crucial for the trustworthy development and deployment of these black-box models. Prior interpretability methods focus on correlation-based measures to attribute model decisions to individual examples. However, these measures are susceptible to noise and spurious correlations encoded in the model during the training phase (e.g., biased inputs, model overfitting, or misspecification). Moreover, this process has proven to result in noisy and unstable attributions that prevent any transparent understanding of the model's behavior. In this paper, we develop a robust interventional-based method grounded by causal analysis to capture cause-effect mechanisms in pre-trained neural networks and their relation to the prediction. Our novel approach relies on path interventions to infer the causal mechanisms within hidden layers and isolate relevant and necessary information (to model prediction), avoiding noisy ones. The result is task-specific causal explanatory graphs that can audit model behavior and express the actual causes underlying its performance. We apply our method to vision models trained on classification tasks. On image classification tasks, we provide extensive quantitative experiments to show that our approach can capture more stable and faithful explanations than standard attribution-based methods. Furthermore, the underlying causal graphs reveal the neural interactions in the model, making it a valuable tool in other applications (e.g., model repair).
New methods for new data? An overview and illustration of quantitative inductive methods for HRM research
Authors: Alain LACROUX
Date: 2023-05-15 00:00:00
Tasks: Data Valuation

"Data is the new oil", in short, data would be the essential source of the ongoing fourth industrial revolution, which has led some commentators to assimilate too quickly the quantity of data to a source of wealth in itself, and consider the development of big data as an quasi direct cause of profit. Human resources management is not escaping this trend, and the accumulation of large amounts of data on employees is perceived by some entrepreneurs as a necessary and sufficient condition for the construction of predictive models of complex work behaviors such as absenteeism or job performance. In fact, the analogy is somewhat misleading: unlike oil, there are no major issues here concerning the production of data (whose flows are generated continuously and at low cost by various information systems), but rather their ''refining'', i.e. the operations necessary to transform this data into a useful product, namely into knowledge. This transformation is where the methodological challenges of data valuation lie, both for practitioners and for academic researchers. Considerations on the methods applicable to take advantage of the possibilities offered by these massive data are relatively recent, and often highlight the disruptive aspect of the current ''data deluge'' to point out that this evolution would be the source of a revival of empiricism in a ''fourth paradigm'' based on the intensive and ''agnostic'' exploitation of massive amounts of data in order to bring out new knowledge, following a purely inductive logic. Although we do not adopt this speculative point of view, it is clear that data-driven approaches are scarce in quantitative HRM studies. However, there are well-established methods, particularly in the field of data mining, which are based on inductive approaches. This area of quantitative analysis with an inductive aim is still relatively unexplored in HRM ( apart from typological analyses). The objective of this paper is first to give an overview of data driven methods that can be used for HRM research, before proposing an empirical illustration which consists in an exploratory research combining a latent profile analysis and an exploration by Gaussian graphical models.
Covariate-distance Weighted Regression (CWR): A Case Study for Estimation of House Prices
Authors: Hone-Jay Chu, Po-Hung Chen, Sheng-Mao Chang, Muhammad Zeeshan Ali, Sumriti Ranjan Patra
Date: 2023-05-15 00:00:00
Tasks: Attribute, regression

Geographically weighted regression (GWR) is a popular tool for modeling spatial heterogeneity in a regression model. However, the current weighting function used in GWR only considers the geographical distance, while the attribute similarity is totally ignored. In this study, we proposed a covariate weighting function that combines the geographical distance and attribute distance. The covariate-distance weighted regression (CWR) is the extension of GWR including geographical distance and attribute distance. House prices are affected by numerous factors, such as house age, floor area, and land use. Prediction model is used to help understand the characteristics of regional house prices. The CWR was used to understand the relationship between the house price and controlling factors. The CWR can consider the geological and attribute distances, and produce accurate estimates of house price that preserve the weight matrix for geological and attribute distance functions. Results show that the house attributes/conditions and the characteristics of the house, such as floor area and house age, might affect the house price. After factor selection, in which only house age and floor area of a building are considered, the RMSE of the CWR model can be improved by 2.9%-26.3% for skyscrapers when compared to the GWR. CWR can effectively reduce estimation errors from traditional spatial regression models and provide novel and feasible models for spatial estimation.
Building Energy Efficiency through Advanced Regression Models and Metaheuristic Techniques for Sustainable Management
Authors: Hamed Khosravi, Hadi Sahebi, Rahim khanizad, Imtiaz Ahmed
Date: 2023-05-15 00:00:00
Tasks: Management, regression

In the context of global sustainability, buildings are significant consumers of energy, emphasizing the necessity for innovative strategies to enhance efficiency and reduce environmental impact. This research leverages extensive raw data from building infrastructures to uncover energy consumption patterns and devise strategies for optimizing resource use. We investigate the factors influencing energy efficiency and cost reduction in buildings, utilizing Lasso Regression, Decision Tree, and Random Forest models for accurate energy use forecasting. Our study delves into the factors affecting energy utilization, focusing on primary fuel and electrical energy, and discusses the potential for substantial cost savings and environmental benefits. Significantly, we apply metaheuristic techniques to enhance the Decision Tree algorithm, resulting in improved predictive precision. This enables a more nuanced understanding of the characteristics of buildings with high and low energy efficiency potential. Our findings offer practical insights for reducing energy consumption and operational costs, contributing to the broader goals of sustainable development and cleaner production. By identifying key drivers of energy use in buildings, this study provides a valuable framework for policymakers and industry stakeholders to implement cleaner and more sustainable energy practices.
Smart Home Energy Management: VAE-GAN synthetic dataset generator and Q-learning
Authors: Mina Razghandi, Hao Zhou, Melike Erol-Kantarci, Damla Turgut
Date: 2023-05-14 00:00:00
Tasks: energy management, Generative Adversarial Network, Management, Q-Learning, Time Series

Recent years have noticed an increasing interest among academia and industry towards analyzing the electrical consumption of residential buildings and employing smart home energy management systems (HEMS) to reduce household energy consumption and costs. HEMS has been developed to simulate the statistical and functional properties of actual smart grids. Access to publicly available datasets is a major challenge in this type of research. The potential of artificial HEMS applications will be further enhanced with the development of time series that represent different operating conditions of the synthetic systems. In this paper, we propose a novel variational auto-encoder-generative adversarial network (VAE-GAN) technique for generating time-series data on energy consumption in smart homes. We also explore how the generative model performs when combined with a Q-learning-based HEMS. We tested the online performance of Q-learning-based HEMS with real-world smart home data. To test the generated dataset, we measure the Kullback-Leibler (KL) divergence, maximum mean discrepancy (MMD), and the Wasserstein distance between the probability distributions of the real and synthetic data. Our experiments show that VAE-GAN-generated synthetic data closely matches the real data distribution. Finally, we show that the generated data allows for the training of a higher-performance Q-learning-based HEMS compared to datasets generated with baseline approaches.