pwc-archive/papers-with-abstracts huggingface dataset:
Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems
Authors: Alexander Ororbia, Ankur Mali
Date: 2022-09-19 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning (RL)

In this article, we propose a backpropagation-free approach to robotic control through the neuro-cognitive computational framework of neural generative coding (NGC), designing an agent built completely from powerful predictive coding/processing circuits that facilitate dynamic, online learning from sparse rewards, embodying the principles of planning-as-inference. Concretely, we craft an adaptive agent system, which we call active predictive coding (ActPC), that balances an internally-generated epistemic signal (meant to encourage intelligent exploration) with an internally-generated instrumental signal (meant to encourage goal-seeking behavior) to ultimately learn how to control various simulated robotic systems as well as a complex robotic arm using a realistic robotics simulator, i.e., the Surreal Robotics Suite, for the block lifting task and can pick-and-place problems. Notably, our experimental results demonstrate that our proposed ActPC agent performs well in the face of sparse (extrinsic) reward signals and is competitive with or outperforms several powerful backprop-based RL approaches.
Application of Neural Network in the Prediction of NOx Emissions from Degrading Gas Turbine
Authors: Zhenkun Zheng, Alan Rezazadeh
Date: 2022-09-19 00:00:00

This paper is aiming to apply neural network algorithm for predicting the process response (NOx emissions) from degrading natural gas turbines. Nine different process variables, or predictors, are considered in the predictive modelling. It is found out that the model trained by neural network algorithm should use part of recent data in the training and validation sets accounting for the impact of the system degradation. R-Square values of the training and validation sets demonstrate the validity of the model. The residue plot, without any clear pattern, shows the model is appropriate. The ranking of the importance of the process variables are demonstrated and the prediction profile confirms the significance of the process variables. The model trained by using neural network algorithm manifests the optimal settings of the process variables to reach the minimum value of NOx emissions from the degrading gas turbine system.
On the Theoretical Properties of Noise Correlation in Stochastic Optimization
Authors: Aurelien Lucchi, Frank Proske, Antonio Orvieto, Francis Bach, Hans Kersting
Date: 2022-09-19 00:00:00
Tasks: Stochastic Optimization

Studying the properties of stochastic noise to optimize complex non-convex functions has been an active area of research in the field of machine learning. Prior work has shown that the noise of stochastic gradient descent improves optimization by overcoming undesirable obstacles in the landscape. Moreover, injecting artificial Gaussian noise has become a popular idea to quickly escape saddle points. Indeed, in the absence of reliable gradient information, the noise is used to explore the landscape, but it is unclear what type of noise is optimal in terms of exploration ability. In order to narrow this gap in our knowledge, we study a general type of continuous-time non-Markovian process, based on fractional Brownian motion, that allows for the increments of the process to be correlated. This generalizes processes based on Brownian motion, such as the Ornstein-Uhlenbeck process. We demonstrate how to discretize such processes which gives rise to the new algorithm fPGD. This method is a generalization of the known algorithms PGD and Anti-PGD. We study the properties of fPGD both theoretically and empirically, demonstrating that it possesses exploration abilities that, in some cases, are favorable over PGD and Anti-PGD. These results open the field to novel ways to exploit noise for training machine learning models.
RESHAPE: Explaining Accounting Anomalies in Financial Statement Audits by enhancing SHapley Additive exPlanations
Authors: Ricardo Müller, Marco Schreyer, Timur Sattarov, Damian Borth
Date: 2022-09-19 00:00:00
Tasks: Attribute, Explainable Artificial Intelligence (XAI)

Detecting accounting anomalies is a recurrent challenge in financial statement audits. Recently, novel methods derived from Deep-Learning (DL) have been proposed to audit the large volumes of a statement's underlying accounting records. However, due to their vast number of parameters, such models exhibit the drawback of being inherently opaque. At the same time, the concealing of a model's inner workings often hinders its real-world application. This observation holds particularly true in financial audits since auditors must reasonably explain and justify their audit decisions. Nowadays, various Explainable AI (XAI) techniques have been proposed to address this challenge, e.g., SHapley Additive exPlanations (SHAP). However, in unsupervised DL as often applied in financial audits, these methods explain the model output at the level of encoded variables. As a result, the explanations of Autoencoder Neural Networks (AENNs) are often hard to comprehend by human auditors. To mitigate this drawback, we propose (RESHAPE), which explains the model output on an aggregated attribute-level. In addition, we introduce an evaluation framework to compare the versatility of XAI methods in auditing. Our experimental results show empirical evidence that RESHAPE results in versatile explanations compared to state-of-the-art baselines. We envision such attribute-level explanations as a necessary next step in the adoption of unsupervised DL techniques in financial auditing.
Physics-Constrained Neural Network for Design and Feature-Based Optimization of Weave Architectures
Authors: Haotian Feng, Sabarinathan P Subramaniyan, Hridyesh Tewani, Pavana Prabhakar
Date: 2022-09-19 00:00:00

Woven fabrics play an essential role in everyday textiles for clothing/sportswear, water filtration, and retaining walls, to reinforcements in stiff composites for lightweight structures like aerospace, sporting, automotive, and marine industries. Several possible combinations of weave patterns and material choices, which comprise weave architecture, present a challenging question about how they could influence the physical and mechanical properties of woven fabrics and reinforced structures. In this paper, we present a novel Physics-Constrained Neural Network (PCNN) to predict the mechanical properties like the modulus of weave architectures and the inverse problem of predicting pattern/material sequence for a design/target modulus value. The inverse problem is particularly challenging as it usually requires many iterations to find the appropriate architecture using traditional optimization approaches. We show that the proposed PCNN can effectively predict weave architecture for the desired modulus with higher accuracy than several baseline models considered. We present a feature-based optimization strategy to improve the predictions using features in the Grey Level Co-occurrence Matrix (GLCM) space. We combine PCNN with this feature-based optimization to discover near-optimal weave architectures to facilitate the initial design of weave architecture. The proposed frameworks will primarily enable the woven composite analysis and optimization process, and be a starting point to introduce Knowledge-guided Neural Networks into the complex structural analysis.
Machine Learning based Extraction of Boundary Conditions from Doppler Echo Images for Patient Specific Coarctation of the Aorta: Computational Fluid Dynamics Study
Authors: Vincent Milimo Masilokwa Punabantu, Malebogo Ngoepe, Amit Kumar Mishra, Thomas Aldersley, John Lawrenson, Liesl Zuhlke
Date: 2022-09-19 00:00:00

Purpose- Coarctation of the Aorta (CoA) patient-specific computational fluid dynamics (CFD) studies in resource constrained settings are limited by the available imaging modalities for geometry and velocity data acquisition. Doppler echocardiography has been seen as a suitable velocity acquisition modality due to its higher availability and safety. This study aimed to investigate the application of classical machine learning (ML) methods to create an adequate and robust approach for obtaining boundary conditions (BCs) from Doppler Echocardiography images, for haemodynamic modeling using CFD. Methods- Our proposed approach combines ML and CFD to model haemodynamic flow within the region of interest. With the key feature of the approach being the use of ML models to calibrate the inlet and outlet boundary conditions (BCs) of the CFD model. The key input variable for the ML model was the patients heart rate as this was the parameter that varied in time across the measured vessels within the study. ANSYS Fluent was used for the CFD component of the study whilst the scikit-learn python library was used for the ML component. Results- We validated our approach against a real clinical case of severe CoA before intervention. The maximum coarctation velocity of our simulations were compared to the measured maximum coarctation velocity obtained from the patient whose geometry is used within the study. Of the 5 ML models used to obtain BCs the top model was within 5\% of the measured maximum coarctation velocity. Conclusion- The framework demonstrated that it was capable of taking variations of the patients heart rate between measurements into account. Thus, enabling the calculation of BCs that were physiologically realistic when the heart rate was scaled across each vessel whilst providing a reasonably accurate solution.
SAMP: A Model Inference Toolkit of Post-Training Quantization for Text Processing via Self-Adaptive Mixed-Precision
Authors: Rong Tian, Zijing Zhao, Weijie Liu, Haoyan Liu, Weiquan Mao, Zhe Zhao, Kan Zhou
Date: 2022-09-19 00:00:00
Tasks: Quantization

The latest industrial inference engines, such as FasterTransformer and TurboTransformers, have verified that half-precision floating point (FP16) and 8-bit integer (INT8) quantization can greatly improve model inference speed. However, the existing INT8 quantization methods are too complicated, and improper usage will lead to model performance damage greatly. In this paper, we develop a toolkit for users to easily quantize their models for inference, in which Self-Adaptive Mixed-Precision (SAMP) is proposed to automatically control quantization rate by a mixed-precision architecture to balance model accuracy and efficiency. Experimental results show that our SAMP toolkit has a higher speedup than PyTorch and FasterTransformer while ensuring the required accuracy. In addition, SAMP is based on a modular design, decoupling the tokenizer, embedding, encoder and target layers, which allows users to handle various downstream tasks and can be seamlessly integrated into PyTorch.
Computing Anti-Derivatives using Deep Neural Networks
Authors: D. Chakraborty, S. Gopalakrishnan
Date: 2022-09-19 00:00:00
Tasks: Form

This paper presents a novel algorithm to obtain the closed-form anti-derivative of a function using Deep Neural Network architecture. In the past, mathematicians have developed several numerical techniques to approximate the values of definite integrals, but primitives or indefinite integrals are often non-elementary. Anti-derivatives are necessarily required when there are several parameters in an integrand and the integral obtained is a function of those parameters. There is no theoretical method that can do this for any given function. Some existing ways to get around this are primarily based on either curve fitting or infinite series approximation of the integrand, which is then integrated theoretically. Curve fitting approximations are inaccurate for highly non-linear functions and require a different approach for every problem. On the other hand, the infinite series approach does not give a closed-form solution, and their truncated forms are often inaccurate. We claim that using a single method for all integrals, our algorithm can approximate anti-derivatives to any required accuracy. We have used this algorithm to obtain the anti-derivatives of several functions, including non-elementary and oscillatory integrals. This paper also shows the applications of our method to get the closed-form expressions of elliptic integrals, Fermi-Dirac integrals, and cumulative distribution functions and decrease the computation time of the Galerkin method for differential equations.
The GenCol algorithm for high-dimensional optimal transport: general formulation and application to barycenters and Wasserstein splines
Authors: Gero Friesecke, Maximilian Penka
Date: 2022-09-19 00:00:00

We extend the recently introduced genetic column generation algorithm for high-dimensional multi-marginal optimal transport from symmetric to general problems. We use the algorithm to calculate accurate mesh-free Wasserstein barycenters and cubic Wasserstein splines.
NIERT: Accurate Numerical Interpolation through Unifying Scattered Data Representations using Transformer Encoder
Authors: Shizhe Ding, Boyang xia, Milong Ren, Dongbo Bu
Date: 2022-09-19 00:00:00
Tasks: Decoder, Representation Learning

Interpolation for scattered data is a classical problem in numerical analysis, with a long history of theoretical and practical contributions. Recent advances have utilized deep neural networks to construct interpolators, exhibiting excellent and generalizable performance. However, they still fall short in two aspects: \textbf{1) inadequate representation learning}, resulting from separate embeddings of observed and target points in popular encoder-decoder frameworks and \textbf{2) limited generalization power}, caused by overlooking prior interpolation knowledge shared across different domains. To overcome these limitations, we present a \textbf{N}umerical \textbf{I}nterpolation approach using \textbf{E}ncoder \textbf{R}epresentation of \textbf{T}ransformers (called \textbf{NIERT}). On one hand, NIERT utilizes an encoder-only framework rather than the encoder-decoder structure. This way, NIERT can embed observed and target points into a unified encoder representation space, thus effectively exploiting the correlations among them and obtaining more precise representations. On the other hand, we propose to pre-train NIERT on large-scale synthetic mathematical functions to acquire prior interpolation knowledge, and transfer it to multiple interpolation domains with consistent performance gain. On both synthetic and real-world datasets, NIERT outperforms the existing approaches by a large margin, i.e., 4.3$\sim$14.3$\times$ lower MAE on TFRD subsets, and 1.7/1.8/8.7$\times$ lower MSE on Mathit/PhysioNet/PTV datasets. The source code of NIERT is available at https://github.com/DingShizhe/NIERT.
Gradual Weisfeiler-Leman: Slow and Steady Wins the Race
Authors: Franka Bause, Nils M. Kriege
Date: 2022-09-19 00:00:00
Tasks: Graph Learning, Isomorphism Testing

The classical Weisfeiler-Leman algorithm aka color refinement is fundamental for graph learning with kernels and neural networks. Originally developed for graph isomorphism testing, the algorithm iteratively refines vertex colors. On many datasets, the stable coloring is reached after a few iterations and the optimal number of iterations for machine learning tasks is typically even lower. This suggests that the colors diverge too fast, defining a similarity that is too coarse. We generalize the concept of color refinement and propose a framework for gradual neighborhood refinement, which allows a slower convergence to the stable coloring and thus provides a more fine-grained refinement hierarchy and vertex similarity. We assign new colors by clustering vertex neighborhoods, replacing the original injective color assignment function. Our approach is used to derive new variants of existing graph kernels and to approximate the graph edit distance via optimal assignments regarding vertex similarity. We show that in both tasks, our method outperforms the original color refinement with only a moderate increase in running time advancing the state of the art.
A cost-based multi-layer network approach for the discovery of patient phenotypes
Authors: Clara Puga, Uli Niemann, Winfried Schlee, Myra Spiliopoulou
Date: 2022-09-19 00:00:00
Tasks: Community Detection

Clinical records frequently include assessments of the characteristics of patients, which may include the completion of various questionnaires. These questionnaires provide a variety of perspectives on a patient's current state of well-being. Not only is it critical to capture the heterogeneity given by these perspectives, but there is also a growing demand for developing cost-effective technologies for clinical phenotyping. Filling out many questionnaires may be a strain for the patients and therefore costly. In this work, we propose COBALT -- a cost-based layer selector model for detecting phenotypes using a community detection approach. Our goal is to minimize the number of features used to build these phenotypes while preserving its quality. We test our model using questionnaire data from chronic tinnitus patients and represent the data in a multi-layer network structure. The model is then evaluated by predicting post-treatment data using baseline features (age, gender, and pre-treatment data) as well as the identified phenotypes as a feature. For some post-treatment variables, predictors using phenotypes from COBALT as features outperformed those using phenotypes detected by traditional clustering methods. Moreover, using phenotype data to predict post-treatment data proved beneficial in comparison with predictors that were solely trained with baseline features.
A Causal Intervention Scheme for Semantic Segmentation of Quasi-periodic Cardiovascular Signals
Authors: Xingyao Wang, Yuwen Li, Hongxiang Gao, Xianghong Cheng, Jianqing Li, Chengyu Liu
Date: 2022-09-19 00:00:00
Tasks: Attribute, Rhythm, Segmentation, Semantic Segmentation

Precise segmentation is a vital first step to analyze semantic information of cardiac cycle and capture anomaly with cardiovascular signals. However, in the field of deep semantic segmentation, inference is often unilaterally confounded by the individual attribute of data. Towards cardiovascular signals, quasi-periodicity is the essential characteristic to be learned, regarded as the synthesize of the attributes of morphology (Am) and rhythm (Ar). Our key insight is to suppress the over-dependence on Am or Ar while the generation process of deep representations. To address this issue, we establish a structural causal model as the foundation to customize the intervention approaches on Am and Ar, respectively. In this paper, we propose contrastive causal intervention (CCI) to form a novel training paradigm under a frame-level contrastive framework. The intervention can eliminate the implicit statistical bias brought by the single attribute and lead to more objective representations. We conduct comprehensive experiments with the controlled condition for QRS location and heart sound segmentation. The final results indicate that our approach can evidently improve the performance by up to 0.41% for QRS location and 2.73% for heart sound segmentation. The efficiency of the proposed method is generalized to multiple databases and noisy signals.
Enforcing the consensus between Trajectory Optimization and Policy Learning for precise robot control
Authors: Quentin Le Lidec, Wilson Jallet, Ivan Laptev, Cordelia Schmid, Justin Carpentier
Date: 2022-09-19 00:00:00
Tasks: Reinforcement Learning (RL), valid

Reinforcement learning (RL) and trajectory optimization (TO) present strong complementary advantages. On one hand, RL approaches are able to learn global control policies directly from data, but generally require large sample sizes to properly converge towards feasible policies. On the other hand, TO methods are able to exploit gradient-based information extracted from simulators to quickly converge towards a locally optimal control trajectory which is only valid within the vicinity of the solution. Over the past decade, several approaches have aimed to adequately combine the two classes of methods in order to obtain the best of both worlds. Following on from this line of research, we propose several improvements on top of these approaches to learn global control policies quicker, notably by leveraging sensitivity information stemming from TO methods via Sobolev learning, and augmented Lagrangian techniques to enforce the consensus between TO and policy learning. We evaluate the benefits of these improvements on various classical tasks in robotics through comparison with existing approaches in the literature.
Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$-Covers
Authors: Sejun Park, Umut Şimşekli, Murat A. Erdogdu
Date: 2022-09-19 00:00:00
Tasks: Generalization Bounds

In this paper, we propose a new covering technique localized for the trajectories of SGD. This localization provides an algorithm-specific complexity measured by the covering number, which can have dimension-independent cardinality in contrast to standard uniform covering arguments that result in exponential dimension dependency. Based on this localized construction, we show that if the objective function is a finite perturbation of a piecewise strongly convex and smooth function with $P$ pieces, i.e. non-convex and non-smooth in general, the generalization error can be upper bounded by $O(\sqrt{(\log n\log(nP))/n})$, where $n$ is the number of data samples. In particular, this rate is independent of dimension and does not require early stopping and decaying step size. Finally, we employ these results in various contexts and derive generalization bounds for multi-index linear models, multi-class support vector machines, and $K$-means clustering for both hard and soft label setups, improving the known state-of-the-art rates.
A novel approach for wafer defect pattern classification based on topological data analysis
Authors: Seungchan Ko, Dowan Koo
Date: 2022-09-19 00:00:00
Tasks: Classification, Management, Topological Data Analysis

In semiconductor manufacturing, wafer map defect pattern provides critical information for facility maintenance and yield management, so the classification of defect patterns is one of the most important tasks in the manufacturing process. In this paper, we propose a novel way to represent the shape of the defect pattern as a finite-dimensional vector, which will be used as an input for a neural network algorithm for classification. The main idea is to extract the topological features of each pattern by using the theory of persistent homology from topological data analysis (TDA). Through some experiments with a simulated dataset, we show that the proposed method is faster and much more efficient in training with higher accuracy, compared with the method using convolutional neural networks (CNN) which is the most common approach for wafer map defect pattern classification. Moreover, our method outperforms the CNN-based method when the number of training data is not enough and is imbalanced.