On spectral methods for variance based sensitivity analysis

Consider a mathematical model with a finite number of random parameters. 
Variance based sensitivity analysis provides a framework to characterize 
the contribution of the individual parameters to the total variance of 
the model response. We consider the spectral methods for variance based 
sensitivity analysis which 
utilize representations of square integrable random variables in a 
generalized polynomial chaos basis. 
Taking a measure theoretic point of view, we 
provide a rigorous and at the same time intuitive perspective on the 
spectral methods for variance based sensitivity analysis. 
Moreover, we discuss approximation errors incurred by fixing 
inessential random 
parameters, when approximating functions with generalized polynomial 
chaos expansions.

Related papers:

title: Importance measures in global sensitivity analysis of nonlinear models
abstract: The present paper deals with a new method of global sensitivity analysis of nonlinear models. This is based on a measure of importance to calculate the fractional contribution of the input parameters to the variance of the model prediction. Measures of importance in sensitivity analysis have been suggested by several authors, whose work is reviewed in this article. More emphasis is given to the developments of sensitivity indices by the Russian mathematician I.M. Sobol'. Given that Sobol' treatment of the measure of importance is the most general, his formalism is employed throughout this paper where conceptual and computational improvements of the method are presented. The computational novelty of this study is the introduction of the ‘total effect’ parameter index. This index provides a measure of the total effect of a given parameter, including all the possible synergetic terms between that parameter and all the others. Rank transformation of the data is also introduced in order to increase the reproducibility of the method. These methods are tested on a few analytical and computer models. The main conclusion of this work is the identification of a sensitivity analysis methodology which is both flexible, accurate and informative, and which can be achieved at reasonable computational cost.

title: Sensitivity analysis for importance assessment.
abstract: We review briefly some examples that would support an extended role for quantitative sensitivity analysis in the context of model-based analysis (Section 1). We then review what features a quantitative sensitivity analysis needs to have to play such a role (Section 2). The methods that meet these requirements are described in Section 3; an example is provided in Section 4. Some pointers to further research are set out in Section 5.

title: Stochastic Finite Elements: A Spectral Approach
abstract: Representation of stochastic processes stochastic finite element method - response representation stochastic finite element method - response statistics numerical examples.

title: The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations
abstract: We present a new method for solving stochastic differential equations based on Galerkin projections and extensions of Wiener's polynomial chaos. Specifically, we represent the stochastic processes with an optimum trial basis from the Askey family of orthogonal polynomials that reduces the dimensionality of the system and leads to exponential convergence of the error. Several continuous and discrete processes are treated, and numerical examples show substantial speed-up compared to Monte Carlo simulations for low dimensional stochastic inputs.

title: Spectral stochastic uncertainty quantification in chemical systems
abstract: Uncertainty quantification (UQ) in the computational modelling of physical systems is important for scientific investigation, engineering design, and model validation. We have implemented an ‘intrusive’ UQ technique in which (1) model parameters and field variables are modelled as stochastic quantities, and are represented using polynomial chaos (PC) expansions in terms of Hermite polynomial functions of Gaussian random variables, and (2) the deterministic model equations are reformulated using Galerkin projection into a set of equations for the time evolution of the field variable PC mode strengths. The mode strengths relate specific parametric uncertainties to their effects on model outputs. In this work, the intrusive reformulation is applied to homogeneous ignition using a detailed chemistry model through the development of a reformulated pseudospectral chemical source term. We present results analysing the growth of uncertainty during the ignition process. We also discuss numerical issues pertaining to the accurate representation of uncertainty with truncated PC expansions, and ensuing stability of the time integration of the chemical system.

title: Multi-Resolution-Analysis Scheme for Uncertainty Quantification in Chemical Systems
abstract: This paper presents a multi-resolution approach for the propagation of parametric uncertainty in chemical systems. It is motivated by previous studies where Galerkin formulations of Wiener-Hermite expansions were found to fail in the presence of steep dependences of the species concentrations with regard to the reaction rates. The multi-resolution scheme is based on representation of the uncertain concentration in terms of compact polynomial multi-wavelets, allowing for the control of the convergence in terms of polynomial order and resolution level. The resulting representation is shown to greatly improve the robustness of the Galerkin procedure in presence of steep dependences. However, this improvement comes with a higher computational cost which drastically increases with the number of uncertain reaction rates. To overcome this drawback an adaptive strategy is proposed to control locally (in the parameter space) and in time the resolution level. The efficiency of the method is demonstrated for an uncertain chemical system having eight random parameters.

title: Uncertainty Quantification and Polynomial Chaos Techniques in Computational Fluid Dynamics
abstract: The quantification of uncertainty in computational fluid dynamics (CFD) predictions is both a significant challenge and an important goal. Probabilistic uncertainty quantification (UQ) methods have been used to propagate uncertainty from model inputs to outputs when input uncertainties are large and have been characterized probabilistically. Polynomial chaos (PC) methods have found increased use in probabilistic UQ over the past decade. This review describes the use of PC expansions for the representation of random variables/fields and discusses their utility for the propagation of uncertainty in computational models, focusing on CFD models. Many CFD applications are considered, including flow in porous media, incompressible and compressible flows, and thermofluid and reacting flows. The review examines each application area, focusing on the demonstrated use of PC UQ and the associated challenges. Cross-cutting challenges with time unsteadiness and long time horizons are also discussed.

title: Spectral Methods for Uncertainty Quantification: With Applications to Computational Fluid Dynamics
abstract: Uncertainty Quantification and Propagation.- Basic  Spectral Expansions.- Non-intrusive Methods.- Galerkin Methods.- Detailed Elementary Applications.- Application to Navier-Stokes Equations.- Advanced topics.- Solvers for Stochastic Galerkin Problems.- Wavelet and Multiresolution Analysis Schemes.- Adaptive Methods.-

title: Spectral Representation and Reduced Order Modeling of the Dynamics of Stochastic Reaction Networks via Adaptive Data Partitioning
abstract: Dynamical analysis tools are well established for deterministic models. However, for many biochemical phenomena in cells the molecule count is low, leading to stochastic behavior that causes deterministic macroscale reaction models to fail. The main mathematical framework representing these phenomena is based on jump Markov processes that model the underlying stochastic reaction network. Conventional dynamical analysis tools do not readily generalize to the stochastic setting due to nondifferentiability and absence of explicit state evolution equations. We developed a reduced order methodology for dynamical analysis that relies on the Karhunen-Loeve decomposition and polynomial chaos expansions. The methodology relies on adaptive data partitioning to obtain an accurate representation of the stochastic process, especially in the case of multimodal behavior. As a result, a mixture model is obtained that represents the reduced order dynamics of the system. The Schlogl model is used as a prototype bistable process that exhibits time scale separation and leads to multimodality in the reduced order model.

title: On the convergence of generalized polynomial chaos expansions
abstract: A number of approaches for discretizing partial differential equations with random data are based on generalized polynomial chaos expansions of random variables. These constitute generalizations of the polynomial chaos expansions introduced by Norbert Wiener to expansions in polynomials orthogonal with respect to non-Gaussian probability measures. We present conditions on such measures which imply mean-square convergence of generalized polynomial chaos expansions to the correct limit and complement these with illustrative examples.

title: The Orthogonal Development of Non-Linear Functionals in Series of Fourier-Hermite Functionals
abstract: (see, for example, Kaczmarz and Steinhaus [I, pp. 143-144]). Let (2.4) tap(t)} p = 1, 2, 3, be any orthonormal set of real functions, each belonging to L2(0, 1). Paley and 1 Wiener [II] have shown for each index p = 1, 2, that f ) dx(t) exist as a generalized Stieltjes integral for almost all functions x(&) of C and that the equality W 1~~~~~~~~~~~~ |G a, ot(t) dx(t), * ,Iap(t) dx(t)] dwx (2.5) c 00 -p/2 L G(ui, * , up)euhu du, ... du.

title: Polynomial chaos expansion for sensitivity analysis
abstract: In this paper, the computation of Sobol's sensitivity indices from the polynomial chaos expansion of a model output involving uncertain inputs is investigated. It is shown that when the model output is smooth with regards to the inputs, a spectral convergence of the computed sensitivity indices is achieved. However, even for smooth outputs the method is limited to a moderate number of inputs, say 10â€“20, as it becomes computationally too demanding to reach the convergence domain. Alternative methods (such as sampling strategies) are then more attractive. The method is also challenged when the output is non-smooth even when the number of inputs is limited.

title: Global sensitivity analysis using polynomial chaos expansions
abstract: Global sensitivity analysis (SA) aims at quantifying the respective effects of input random variables (or combinations thereof) onto the variance of the response of a physical or mathematical model. Among the abundant literature on sensitivity measures, the Sobol’ indices have received much attention since they provide accurate information for most models. The paper introduces generalized polynomial chaos expansions (PCE) to build surrogate models that allow one to compute the Sobol’ indices analytically as a post-processing of the PCE coefficients. Thus the computational cost of the sensitivity indices practically reduces to that of estimating the PCE coefficients. An original non intrusive regression-based approach is proposed, together with an experimental design of minimal size. Various application examples illustrate the approach, both from the field of global SA (i.e. well-known benchmark problems) and from the field of stochastic mechanics. The proposed method gives accurate results for various examples that involve up to eight input random variables, at a computational cost which is 2–3 orders of magnitude smaller than the traditional Monte Carlo-based evaluation of the Sobol’ indices.

title: Estimating the approximation error when fixing unessential factors in global sensitivity analysis
abstract: One of the major settings of global sensitivity analysis is that of fixing non-influential factors, in order to reduce the dimensionality of a model. However, this is often done without knowing the magnitude of the approximation error being produced. This paper presents a new theorem for the estimation of the average approximation error generated when fixing a group of non-influential factors. A simple function where analytical solutions are available is used to illustrate the theorem. The numerical estimation of small sensitivity indices is discussed.

title: Foundations of modern probability
abstract: * Measure Theory-Basic Notions * Measure Theory- Processes, Distributions, and Independence * Random Sequences, Series, and Averages * Characteristic Functions and Classical Limit Theorems * Conditioning and Disintegration * Martingales and Optional Times * Markov Processes and Discrete-Time Chains * Random Walks and Renewal Theory * Stationary Processes and Ergodic Theory * Special Notions of Symmetry and Invariance * Poisson and Pure Jump-Type Markov Processes * Gaussian Processes and Brownian Motion * Skorohod Embedding and Invariance Principles * Independent Increments and Infinite Divisibility * Convergence of Random Processes, Measures, and Sets * Stochastic Integrals and Quadratic Variation * Continuous Martingales and Brownian Motion * Feller Processes and Semigroups * Ergodic Properties of Markov Processes * Stochastic Differential Equations and Martingale Problems * Local Time, Excursions, and Additive Functionals * One-Dimensional SDEs and Diffusions * Connections with PDEs and Potential Theory * Predictability, Compensation, and Excessive Functions * Semimartingales and General Stochastic Integration * Large Deviations *   Some Special Spaces * Historical and Bibliographical Notes * Bibliography * Indices

title: Measure Theory and Probability Theory
abstract: This is a graduate level textbook on measure theory and probability theory. It presents the main concepts and results in measure theory and probability theory in a simple and easy-to-understand way. It further provides heuristic explanations behind the theory to help students see the big picture. The book can be used as a text for a two semester sequence of courses in measure theory and probability theory, with an option to include supplemental material on stochastic processes and special topics. Prerequisites are kept to the minimal level and the book is intended primarily for first year Ph.D. students in mathematics and statistics.

title: Global Sensitivity Analysis: The Primer
abstract: Preface. 1.  1.1 Models and Sensitivity Analysis. 1.1.1 Definition. 1.1.2 Models. 1.1.3 Models and Uncertainty. 1.1.4 How to Set Up Uncertainty and Sensitivity Analyses. 1.1.5 Implications for Model Quality. 1.2 Methods and Settings for Sensitivity Analysis - An Introduction. 1.2.1 Local versus Global. 1.2.2 A Test Model. 1.2.3 Scatterplots versus Derivatives. 1.2.4 Sigma-normalized Derivatives. 1.2.5 Monte Carlo and Linear Regression. 1.2.6 Conditional Variances - First Path. 1.2.7 Conditional Variances - Second Path. 1.2.8 Application to Model (1.3). 1.2.9 A First Setting: 'Factor Prioritization' 1.2.10 Nonadditive Models. 1.2.11 Higher-order Sensitivity Indices. 1.2.12 Total Effects. 1.2.13 A Second Setting: 'Factor Fixing'. 1.2.14 Rationale for Sensitivity Analysis.   Possible Pitfalls for a Sensitivity Analysis.  Concluding Remarks. 1.6 Exercises. 1.7 Answers.   Dependency on a Single Parameter. 2.3 Sensitivity Analysis of a Single Parameter. 2.3.1 Random Values. 2.3.2 Stratified Sampling. 2.3.3 Mean and Variance Estimates for Stratified Sampling. 2.4 Sensitivity Analysis of Multiple Parameters. 2.4.1 Linear Models. 2.4.2 One-at-a-time (OAT) Sampling. 2.4.3 Limits on the Number of Influential Parameters. 2.4.4 Fractional Factorial Sampling. 2.4.5 Latin Hypercube Sampling. 2.4.6 Multivariate Stratified Sampling. 2.4.7 Quasi-random Sampling with Low-discrepancy Sequences. 2.5 Group Sampling. 2.6 Exercises. 2.7 Exercise Solutions. 3. Elementary Effects Method.  Working with Groups. 3.6 The EE Method Step by Step.  3.8 Exercises. 3.9 Solutions. 4. Variance-based Methods. 4.1 Different Tests for Different Settings. 4.2 Why Variance? 4.3 Variance-based Methods. A Brief History.   Putting the Method to Work: the Infection Dynamics Model. 4.9 Caveats. 4.10 Exercises. 5. Factor Mapping and Metamodelling.  5.2 Monte Carlo Filtering (MCF). 5.2.1 Implementation of Monte Carlo Filtering. 5.2.2 Pros and Cons. 5.2.3 Exercises. 5.2.4 Solutions. 5.2.5 Examples. 5.3 Metamodelling and the High-Dimensional Model Representation. 5.3.1 Estimating HDMRs and Metamodels. 5.3.2 A Simple Example. 5.3.3 Another Simple Example. 5.3.4 Exercises. 5.3.5 Solutions to Exercises.  from Theory to Practice. 6.1 Example 1: a Composite Indicator. 6.1.1  A Composite Indicator Measuring Countries' Performance in Environmental Sustainability.  Experiment and its Results.  Importance of Jumps in Pricing Options. 6.2.1 Setting the Problem.  The Heston Stochastic Volatility Model with Jumps. 6.2.3 Selecting a Suitable Sensitivity Analysis Method. 6.2.4 The Sensitivity Analysis Experiment. 6.2.5 Conclusions. 6.3 Example 3: a Chemical Reactor. 6.3.1  The Sensitivity Analysis Experiment and its Results.  a Mixed Uncertainty-Sensitivity Plot. 6.4.1 In Brief. 6.5 When to use What? Afterword. Bibliography. Index.

title: Preconditioned Bayesian Regression for Stochastic Chemical Kinetics
abstract: We develop a preconditioned Bayesian regression method that enables sparse polynomial chaos representations of noisy outputs for stochastic chemical systems with uncertain reaction rates. The approach is based on the definition of an appropriate multiscale transformation of the state variables coupled with a Bayesian regression formalism. This enables efficient and robust recovery of both the transient dynamics and the corresponding noise levels. Implementation of the present approach is illustrated through applications to a stochastic Michaelis–Menten dynamics and a higher dimensional example involving a genetic positive feedback loop. In all cases, a stochastic simulation algorithm (SSA) is used to compute the system dynamics. Numerical experiments show that Bayesian preconditioning algorithms can simultaneously accommodate large noise levels and large variability with uncertain parameters, and that robust estimates can be obtained with a small number of SSA realizations.