pwc-archive/papers-with-abstracts huggingface dataset:
Organizational economic sustainability via process optimization and human capital: a Soft Systems Methodology (SSM) approach
Authors: Wadim Strielkowski, Evgeny Kuzmin, Arina Suvorova, Natalya Nikitina, Olga Gorlova
Date: 2023-11-29 00:00:00
Tasks: Articles, Management

This review paper focuses on enhancing organizational economic sustainability through process optimization and human capital effective management utilizing the soft systems methodology (SSM) approach which offers a holistic approach for understanding complex real-world challenges. By emphasizing systems thinking and engaging diverse stakeholders in problem-solving, SSM provides a comprehensive understanding of the problem's context and potential solutions. The approach guides a systematic process of inquiry that leads to feasible and desirable changes in tackling complex problems effectively. Our paper employs the bibliometric analysis based on the sample of 5171 research articles, proceedings papers, and book chapters indexed in Web of Science (WoS) database. We carry out the network cluster analysis using the text data and the bibliometric data with the help of VOSViewer software. Our results confirm that as the real-world situations are becoming more complex and the new challenges such as the global warming and climate change are threatening many economic and social processes, SSM approach is currently getting back at the forefront of academic research related to such topics as organizational management and sustainable human capital efficiency.
Adapting OpenAI's Whisper for Speech Recognition on Code-Switch Mandarin-English SEAME and ASRU2019 Datasets
Authors: Yuhang Yang, Yizhou Peng, Xionghu Zhong, Hao Huang, Eng Siong Chng
Date: 2023-11-29 00:00:00
Tasks: speech-recognition, Speech Recognition

This paper details the experimental results of adapting the OpenAI's Whisper model for Code-Switch Mandarin-English Speech Recognition (ASR) on the SEAME and ASRU2019 corpora. We conducted 2 experiments: a) using adaptation data from 1 to 100/200 hours to demonstrate effectiveness of adaptation, b) examining different language ID setup on Whisper prompt. The Mixed Error Rate results show that the amount of adaptation data may be as low as $1\sim10$ hours to achieve saturation in performance gain (SEAME) while the ASRU task continued to show performance with more adaptation data ($>$100 hours). For the language prompt, the results show that although various prompting strategies initially produce different outcomes, adapting the Whisper model with code-switch data uniformly improves its performance. These results may be relevant also to the community when applying Whisper for related tasks of adapting to new target domains.
Turbocharge Speech Understanding with Pilot Inference
Authors: Rongxiang Wang, Felix Xiaozhu Lin
Date: 2023-11-22 00:00:00
Tasks: Decoder

Modern speech understanding (SU) runs a sophisticated pipeline: ingesting streaming voice input, the pipeline executes encoder-decoder based deep neural networks repeatedly; by doing so, the pipeline generates tentative outputs (called hypotheses), and periodically scores the hypotheses. This paper sets to accelerate SU on resource-constrained edge devices. It takes a hybrid approach: to speed up on-device execution; to offload inputs that are beyond the device's capacity. While the approach is well-known, we address SU's unique challenges with novel techniques: (1) late contextualization, which executes a model's attentive encoder in parallel to the input ingestion; (2) pilot inference, which mitigates the SU pipeline's temporal load imbalance; (3) autoregression offramps, which evaluate offloading decisions based on pilot inferences and hypotheses. Our techniques are compatible with existing speech models, pipelines, and frameworks; they can be applied independently or in combination. Our prototype, called PASU, is tested on Arm platforms with 6 - 8 cores: it delivers SOTA accuracy; it reduces the end-to-end latency by 2x and reduces the offloading needs by 2x.
Fault-Tolerant Four-Dimensional Constellation for Coherent Optical Transmission Systems
Authors: Jingtian Liu, Élie Awwad, Yves Jaouën
Date: 2023-11-29 00:00:00
Tasks: Quantization

We propose a 4-dimensional 2-ary amplitude ring-switched modulation format with 64 symbols, which is denoted as 4D-2A-RS64 encoded over two polarization tributaries to improve the transmission performance over long-haul optical fibers in the presence of the non-linear Kerr effect. At a spectral efficiency of 6 bits per 4D, simulation results show that this format outperforms the polarization division multiplexed (PDM) 8QAM-star modulation as well as the 4D-2A-8PSK over links without inline dispersion management. We evaluate the performance for a WDM transmission of $11\times90~\mathrm{Gbaud}$ channels over a multi-span SSMF link. For an achievable information rate of $4.8\mathrm{bit/s/Hz}$, the maximum transmission distance is improved by $10.6\%$ (400 km) and $4\%$ (160 km) compared to PDM-8QAM-star and 4D-2A-8PSK respectively. The achieved gains are composed of a linear part and a non-linear part, respectively from the improved Euclidean-distance distribution and the constant power property of the 4D modulation. The geometric shaping of the proposed scheme is easy to implement and is robust to Mach-Zehnder modulator (MZM) imbalances and quantization errors stemming from the finite digital-to-analog converter (DAC) resolution. This robustness is compared to the one of other geometric-shaped non-linearity tolerant 4D schemes such as the 4D-2A-8PSK and the 4D-64PRS that can be both outperformed by our scheme in severe conditions.
A Modulation-Format Dependent Closed-form Expression for the Gaussian Noise Model in the Presence of Raman Amplification
Authors: H. Buglia, M. Jarmolovicius, L. Galdino, R. I. Killey, P. Bayvel
Date: 2023-09-13 00:00:00
Tasks: Form

A closed-form expression that estimates the nonlinear interference of arbitrary modulation formats in Raman amplified links is presented. Accounting for any pumping schemes and inter-channel stimulated Raman scattering effect, the formula is applied to an optical bandwidth of 20~THz and validated using numerical simulations.
Exponential Utility Maximization in a Continuous Time Gaussian Framework
Authors: Yan Dolinsky
Date: 2023-11-28 00:00:00

In this work we study the continuous time exponential utility maximization problem in the framework of an investor who is informed about the price changes with a delay. This leads to a non-Markovian stochastic control problem. In the case where the risky asset is given by a Gaussian process (with some additional properties) we establish a solution for the optimal control and the corresponding value. Our approach is purely probabilistic and is based on the theory for Radon-Nikodym derivatives of Gaussian measures developed by Shepp \cite{S:66}, Hitsuda \cite{H:68} and received a new and unifying angle in [2].
Zero-One Laws of Graph Neural Networks
Date: 2023-09-21 00:00:00
Proceeding: NeurIPS 2023 11

Graph neural networks (GNNs) are the de facto standard deep learning architectures for machine learning on graphs.  This has led to a large body of work analyzing the capabilities and limitations of these models,  particularly pertaining to their representation and extrapolation capacity.  We offer a novel theoretical perspective on the representation and extrapolation capacity of GNNs, by answering the question: how do GNNs behave as the number of graph nodes become very large? Under mild assumptions, we show that when we draw graphs of increasing size from the Erdős–Rényi model, the probability that such graphs are mapped to a particular output by a class of GNN classifiers tends to either zero or one. This class includes the popular graph convolutional network architecture. The result establishes `zero-one laws' for these GNNs, and analogously to other convergence laws,  entails theoretical limitations on their capacity.  We empirically verify our results, observing that the theoretical asymptotic limits are evident already on relatively small graphs.
Generalised f-Mean Aggregation for Graph Neural Networks
Date: 2023-09-21 00:00:00
Proceeding: NeurIPS 2023 11

Graph Neural Network (GNN) architectures are defined by their implementations of update and aggregation modules. While many works focus on new ways to parametrise the update modules, the aggregation modules receive comparatively little attention. Because it is difficult to parametrise aggregation functions, currently most methods select a ``standard aggregator'' such as mean, sum, or max. While this selection is often made without any reasoning, it has been shown that the choice in aggregator has a significant impact on performance, and the best choice in aggregator is problem-dependent. Since aggregation is a lossy operation, it is crucial to select the most appropriate aggregator in order to minimise information loss. In this paper, we present GenAgg, a generalised aggregation operator, which parametrises a function space that includes all standard aggregators. In our experiments, we show that GenAgg is able to represent the standard aggregators with much higher accuracy than baseline methods. We also show that using GenAgg as a drop-in replacement for an existing aggregator in a GNN often leads to a significant boost in performance across various tasks.
Error Bounds for Learning with Vector-Valued Random Features
Authors: Samuel Lanthaler, Nicholas H. Nelsen
Date: 2023-05-26 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: regression

This paper provides a comprehensive error analysis of learning with vector-valued random features (RF). The theory is developed for RF ridge regression in a fully general infinite-dimensional input-output setting, but nonetheless applies to and improves existing finite-dimensional analyses. In contrast to comparable work in the literature, the approach proposed here relies on a direct analysis of the underlying risk functional and completely avoids the explicit RF ridge regression solution formula in terms of random matrices. This removes the need for concentration results in random matrix theory or their generalizations to random operators. The main results established in this paper include strong consistency of vector-valued RF estimators under model misspecification and minimax optimal convergence rates in the well-specified setting. The parameter complexity (number of random features) and sample complexity (number of labeled data) required to achieve such rates are comparable with Monte Carlo intuition and free from logarithmic factors.
Identifiability Guarantees for Causal Disentanglement from Soft Interventions
Authors: JiaQi Zhang, Chandler Squires, Kristjan Greenewald, Akash Srivastava, Karthikeyan Shanmugam, Caroline Uhler
Date: 2023-07-12 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Disentanglement

Causal disentanglement aims to uncover a representation of data using latent variables that are interrelated through a causal model. Such a representation is identifiable if the latent model that explains the data is unique. In this paper, we focus on the scenario where unpaired observational and interventional data are available, with each intervention changing the mechanism of a latent variable. When the causal variables are fully observed, statistically consistent algorithms have been developed to identify the causal model under faithfulness assumptions. We here show that identifiability can still be achieved with unobserved causal variables, given a generalized notion of faithfulness. Our results guarantee that we can recover the latent causal model up to an equivalence class and predict the effect of unseen combinations of interventions, in the limit of infinite data. We implement our causal disentanglement framework by developing an autoencoding variational Bayes algorithm and apply it to the problem of predicting combinatorial perturbation effects in genomics.
Adaptive Linear Estimating Equations
Authors: Mufang Ying, Koulik Khamaru, Cun-Hui Zhang
Date: 2023-07-14 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Multi-Armed Bandits

Sequential data collection has emerged as a widely adopted technique for enhancing the efficiency of data gathering processes. Despite its advantages, such data collection mechanism often introduces complexities to the statistical inference procedure. For instance, the ordinary least squares (OLS) estimator in an adaptive linear regression model can exhibit non-normal asymptotic behavior, posing challenges for accurate inference and interpretation. In this paper, we propose a general method for constructing debiased estimator which remedies this issue. It makes use of the idea of adaptive linear estimating equations, and we establish theoretical guarantees of asymptotic normality, supplemented by discussions on achieving near-optimal asymptotic variance. A salient feature of our estimator is that in the context of multi-armed bandits, our estimator retains the non-asymptotic performance of the least square estimator while obtaining asymptotic normality property. Consequently, this work helps connect two fruitful paradigms of adaptive inference: a) non-asymptotic inference using concentration inequalities and b) asymptotic inference via asymptotic normality.
Interaction Measures, Partition Lattices and Kernel Tests for High-Order Interactions
Authors: Zhaolu Liu, Robert L. Peach, Pedro A. M. Mediano, Mauricio Barahona
Date: 2023-06-01 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Computational Efficiency

Models that rely solely on pairwise relationships often fail to capture the complete statistical structure of the complex multivariate data found in diverse domains, such as socio-economic, ecological, or biomedical systems. Non-trivial dependencies between groups of more than two variables can play a significant role in the analysis and modelling of such systems, yet extracting such high-order interactions from data remains challenging. Here, we introduce a hierarchy of $d$-order ($d \geq 2$) interaction measures, increasingly inclusive of possible factorisations of the joint probability distribution, and define non-parametric, kernel-based tests to establish systematically the statistical significance of $d$-order interactions. We also establish mathematical links with lattice theory, which elucidate the derivation of the interaction measures and their composite permutation tests; clarify the connection of simplicial complexes with kernel matrix centring; and provide a means to enhance computational efficiency. We illustrate our results numerically with validations on synthetic data, and through an application to neuroimaging data.
A Novel Framework for Policy Mirror Descent with General Parameterization and Linear Convergence
Authors: Carlo Alfano, Rui Yuan, Patrick Rebeschini
Date: 2023-01-30 00:00:00
Proceeding: NeurIPS 2023 11

Modern policy optimization methods in reinforcement learning, such as TRPO and PPO, owe their success to the use of parameterized policies. However, while theoretical guarantees have been established for this class of algorithms, especially in the tabular setting, the use of general parameterization schemes remains mostly unjustified. In this work, we introduce a novel framework for policy optimization based on mirror descent that naturally accommodates general parameterizations. The policy class induced by our scheme recovers known classes, e.g., softmax, and generates new ones depending on the choice of mirror map. Using our framework, we obtain the first result that guarantees linear convergence for a policy-gradient-based method involving general parameterization. To demonstrate the ability of our framework to accommodate general parameterization schemes, we provide its sample complexity when using shallow neural networks, show that it represents an improvement upon the previous best results, and empirically validate the effectiveness of our theoretical claims on classic control tasks.
Posterior Contraction Rates for Matérn Gaussian Processes on Riemannian Manifolds
Authors: Paul Rosa, Viacheslav Borovitskiy, Alexander Terenin, Judith Rousseau
Date: 2023-09-19 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Gaussian Processes, Uncertainty Quantification

Gaussian processes are used in many machine learning applications that rely on uncertainty quantification. Recently, computational tools for working with these models in geometric settings, such as when inputs lie on a Riemannian manifold, have been developed. This raises the question: can these intrinsic models be shown theoretically to lead to better performance, compared to simply embedding all relevant quantities into $\mathbb{R}^d$ and using the restriction of an ordinary Euclidean Gaussian process? To study this, we prove optimal contraction rates for intrinsic Mat\'ern Gaussian processes defined on compact Riemannian manifolds. We also prove analogous rates for extrinsic processes using trace and extension theorems between manifold and ambient Sobolev spaces: somewhat surprisingly, the rates obtained turn out to coincide with those of the intrinsic processes, provided that their smoothness parameters are matched appropriately. We illustrate these rates empirically on a number of examples, which, mirroring prior work, show that intrinsic processes can achieve better performance in practice. Therefore, our work shows that finer-grained analyses are needed to distinguish between different levels of data-efficiency of geometric Gaussian processes, particularly in settings which involve small data set sizes and non-asymptotic behavior.
Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inference
Authors: Licong Lin, Mufang Ying, Suvrojit Ghosh, Koulik Khamaru, Cun-Hui Zhang
Date: 2023-10-01 00:00:00
Proceeding: NeurIPS 2023 11

Estimation and inference in statistics pose significant challenges when data are collected adaptively. Even in linear models, the Ordinary Least Squares (OLS) estimator may fail to exhibit asymptotic normality for single coordinate estimation and have inflated error. This issue is highlighted by a recent minimax lower bound, which shows that the error of estimating a single coordinate can be enlarged by a multiple of $\sqrt{d}$ when data are allowed to be arbitrarily adaptive, compared with the case when they are i.i.d. Our work explores this striking difference in estimation performance between utilizing i.i.d. and adaptive data. We investigate how the degree of adaptivity in data collection impacts the performance of estimating a low-dimensional parameter component in high-dimensional linear models. We identify conditions on the data collection mechanism under which the estimation error for a low-dimensional parameter component matches its counterpart in the i.i.d. setting, up to a factor that depends on the degree of adaptivity. We show that OLS or OLS on centered data can achieve this matching error. In addition, we propose a novel estimator for single coordinate inference via solving a Two-stage Adaptive Linear Estimating equation (TALE). Under a weaker form of adaptivity in data collection, we establish an asymptotic normality property of the proposed estimator.
MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting
Authors: Felix Biggs, Antonin Schrab, Arthur Gretton
Date: 2023-06-14 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Two-sample testing

We propose novel statistics which maximise the power of a two-sample test based on the Maximum Mean Discrepancy (MMD), by adapting over the set of kernels used in defining it. For finite sets, this reduces to combining (normalised) MMD values under each of these kernels via a weighted soft maximum. Exponential concentration bounds are proved for our proposed statistics under the null and alternative. We further show how these kernels can be chosen in a data-dependent but permutation-independent way, in a well-calibrated test, avoiding data splitting. This technique applies more broadly to general permutation-based MMD testing, and includes the use of deep kernels with features learnt using unsupervised models such as auto-encoders. We highlight the applicability of our MMD-FUSE test on both synthetic low-dimensional and real-world high-dimensional data, and compare its performance in terms of power against current state-of-the-art kernel tests.
Conditional independence testing under misspecified inductive biases
Authors: Felipe Maia Polo, Yuekai Sun, Moulinath Banerjee
Date: 2023-07-05 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: regression

Conditional independence (CI) testing is a fundamental and challenging task in modern statistics and machine learning. Many modern methods for CI testing rely on powerful supervised learning methods to learn regression functions or Bayes predictors as an intermediate step; we refer to this class of tests as regression-based tests. Although these methods are guaranteed to control Type-I error when the supervised learning methods accurately estimate the regression functions or Bayes predictors of interest, their behavior is less understood when they fail due to misspecified inductive biases; in other words, when the employed models are not flexible enough or when the training algorithm does not induce the desired predictors. Then, we study the performance of regression-based CI tests under misspecified inductive biases. Namely, we propose new approximations or upper bounds for the testing errors of three regression-based tests that depend on misspecification errors. Moreover, we introduce the Rao-Blackwellized Predictor Test (RBPT), a regression-based CI test robust against misspecified inductive biases. Finally, we conduct experiments with artificial and real data, showcasing the usefulness of our theory and methods.
Kernel Stein Discrepancy thinning: a theoretical perspective of pathologies and a practical fix with regularization
Authors: Clément Bénard, Brian Staber, Sébastien da Veiga
Date: 2023-01-31 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Bayesian Inference

Stein thinning is a promising algorithm proposed by (Riabiz et al., 2022) for post-processing outputs of Markov chain Monte Carlo (MCMC). The main principle is to greedily minimize the kernelized Stein discrepancy (KSD), which only requires the gradient of the log-target distribution, and is thus well-suited for Bayesian inference. The main advantages of Stein thinning are the automatic remove of the burn-in period, the correction of the bias introduced by recent MCMC algorithms, and the asymptotic properties of convergence towards the target distribution. Nevertheless, Stein thinning suffers from several empirical pathologies, which may result in poor approximations, as observed in the literature. In this article, we conduct a theoretical analysis of these pathologies, to clearly identify the mechanisms at stake, and suggest improved strategies. Then, we introduce the regularized Stein thinning algorithm to alleviate the identified pathologies. Finally, theoretical guarantees and extensive experiments show the high efficiency of the proposed algorithm. An implementation of regularized Stein thinning as the kernax library in python and JAX is available at https://gitlab.com/drti/kernax.