# Presentation: 688897

## Identify Parameters Important to Predictions using PPR&  Identify Existing Observation Locations Important to Predictions using OPR

## PPR Statistics for Exercise 8.1c

- Files are provided for 2 analyses :
- MODE=PPR, PARGROUPS=NO – If we could obtain data on any one parameter, which should it be?
- MODE=PPR, PARGROUPS=YES, 2 parameters per group – If we could obtain data on any pair of parameters, which should they be?

## PPR – Exercise 8.1c

**PPR – Exercise 8.1c**

- Prediction is the advective transport at 100 years travel time.
- PercentReduc=10
- What if we could collect data to reduce by 10 percent the parameter standard deviation?

- PPR = percent decrease in the standard deviation of a prediction produced by a 10-percent decrease in the standard deviation of the parameter.
- Results for the advective-transport predictions at 100 years are shown in next slides:
- First – individual parameters
- Second – pairs of parameters

- x

- y

- Figure 8.15b, p. 210

## Exercise 8.1c: PPR Individual Parameters

- Which parameters rank as most important to the predictions by the ppr statistic?
- With CSS and PSS, HK_2 and POR1&2 were ranked first.
- Why the difference for POR1&2???

- Average ppr statistic for all predictions

- Figure 8.9a, p. 201

**1**

**2**

**3**

## Exercise 8.1c: PPR Individual Parameters

- Changes in meters are small for A100z compared to A100x & A100y. But the vertical dimension is much smaller. PPR correctly represents the different dimensions.

- PPR

- Change,in meters

**Exercise 8.1c: PPR Individual Parameters**

- Figure 8.9b, p. 201

- Figure 8.9c, p. 201

## Exercise 8.1c: PPR Grouped Parameters

- Which parameter pairs would be most beneficial to simultaneously investigate?

- Figure 8.9d, p. 201

- Any pair of:
- HK_1             RCH_1
- VK_CB          RCH_2
- HK_2
- Kind of surprising!

## How is PPR calculated???

- OPR and PPR statistics are based on the calculation of prediction standard deviation, a measure of prediction uncertainty

## Predictions – Advective Travel

- Prediction
- UCODE_2005 can compute the sensitivity of the predicted travel path in three directions:
*X   -   East-West*
*Y   -   North-South*
*Z   -   Up-Down*
- Using calculations described later, the variance and / or standard deviation of predictions can be determined

**Advective path**

## Predictions – Uncertainty

- Standard Deviation
- Measure of spread of values for a variable
- Involves assumptions
- Used in OPR & PPR statistics as a means for comparing relative predictive uncertainty
- The black curve presents the standard deviation in the context of a normal distribution, which may or not be the appropriate distribution for this uncertainty.

- Normal distribution

**Advective path**

## Predictions – Uncertainty

**Predictions – Uncertainty**

- Standard Deviation
- With additional information on parameters or with additional observations – predictive standard deviation is reduced
- Red bars illustrates ‘new’ predictive  standard deviation
- The change in standard deviation makes the probability distribution more narrow.
- Use the difference between the red and the black bars to measure the worth of the additional data

- Normal distribution

- Advective path

- Normal distribution

## Predictions – Uncertainty

**Predictions – Uncertainty**

- Standard Deviation
- With the omission of information about one or more observations – predictive standard deviation is increased
- Red bars illustrate ‘new’ predictive  standard deviation
- The change in standard deviation makes the probability distribution wider.
- Use the difference between the red and the black bars to measure the worth of the omitted data

- Normal distribution

- Advective path

## Standard deviation of a prediction

- standard deviation of the th simulated prediction, z’
- calculated error variance from regression
- vector of prediction sensitivities
-     to parameters
- matrix of observation sensitivities to parameters
- matrix of weights on observations and prior
- transpose the matrix
- parameter variance-covariance matrix

- sz’
- s2
- z’
- _b__ _
- _X _
_**_
- **
- _V_(_b_)_ _

- sz’ = [s2 (       (_X_T___X_)-1            )]1/2

- _V_(_b_) = s2(_X_T___X_)-1

- z’
- _b_  

- z’T
- _b_  

## Standard deviation of a prediction

- All terms in this equation are already available
- weight matrix includes weights on observations and on prior information about parameters
- sensitivity matrix _X_ contains the sensitivities for simulated equivalents to the observations, and entries for prior information on parameters
- First order second moment (FOSM) method
- First order – linearise using first order Taylor’s series
- Second moment – variances and standard deviations
- For OPR and PPR statistics, manipulate __ and _X_

- sz’ = [s2 (       (_X_T___X_)-1            )]1/2

- z’
- _b_  

- z’T
- _b_  

## Standard deviation of a prediction

- All terms in this equation are already available
- weight matrix includes weights on observations and on prior information about parameters
- sensitivity matrix _X_ contains the sensitivities for simulated equivalents to the observations, and entries for prior information on parameters
- First order second moment (FOSM) method
- First order – linearise using first order Taylor’s series
- Second moment – variances and standard deviations
- For OPR and PPR statistics, manipulate __ and _X_

- sz’ = [s2 (       (_X_T___X_)-1            )]1/2

- z’
- _b_  

- z’T
- _b_  

## X and 

- _X_

**Observation part**

**Sensitivities**

**Weighting**

**Prior information part**

## OPR and PPR Statistics - Theory

**OPR and PPR Statistics - Theory**

- _V_  = s2(_X_T ___X_  )-1

- (****j)

- (****j)

- (****j)

- (****j)

- (****j)

- Notation (j) indicates that one of the following has taken place:
- Information has been added regarding parameter(s) j (+)
- Information has been added regarding observation(s) j (+)
- Information has been omitted regarding observation(s) j (-)

- z’
- _b_  

- z’T
- _b_  

- (****j)

- (****j)

- (****j)

## X and 

- _X_

**Observation part**

**Sensitivities**

**Weighting**

**Prior information part**

**For PPR add Prior Information terms**

**For OPR add or remove observation terms**

## OPR and PPR Statistics - Approach

- Calculate hypothetical prediction standard deviation assuming changes in information about parameters or changes to the available observations
- The Parameter-Prediction (PPR) Statistic:
- Evaluate worth of potential new knowledge about parameters, posed in the form of prior information - add to calculations
- The Observation-Prediction (OPR) Statistic:
- Evaluate existing observation locations - omit from calculations
- Evaluate potential new observation locations – add to calculations

**OPR and PPR Statistics - Approach**

## OPR-PPR Program

- Encapsulates OPR and PPR statistics:
- Compatible with the JUPITER API and UCODE_2005
- Distributed with MF2K2DX that will convert MODFLOW-2000 and MODFLOW-2005 output files into the Data-Exchange Files needed by OPR-PPR ***ask Matt
- Tonkin, Tiedeman, Ely, Hill (2007) Documentation for OPR-PPR, USGS Techniques & Methods 6-E2
- Exercise uses the OPR and PPR methods together with the synthetic model

## PPR Statistic Calculation

- The PPR statistic is defined as the percent change in prediction standard deviation caused by increased knowledge about the parameter
- Therefore it measures the relative importance to a prediction of potential new information on a parameter

-  ppr = [1- (sz   / sz)] x 100

- (j)

- (j)

## PPR Statistic - Theory

- Focusing on __ppr:
- Weights on the potential new information are ideally proportional to the uncertainty in that information
- But, it is not known how certain this information will be
- This is overcome pragmatically by calculating the weight that that reduces the ***parameter*** standard deviation by a user specified percentage.

- Weights on existing observations and prior

- Weights on potential new information on parameters

- (j)

## PPR Statistic - Theory

- Calculating weights on potential new information:
- User specifies the desired percent reduction (‘PercentReduc’) in the parameter standard deviation
- Within OPR-PPR:
- Add a nominal initial weight into the weight matrix __ppr for the corresponding parameter
- Iteratively solve the equations above until the standard deviation in that parameter is reduced by the user-specified amount
- Calculate sz

## OPR Statistic Calculation

- The OPR statistic is defined as the percent change in prediction standard deviation caused by:
-     the addition of one or more observations  –  OPR-ADD
- 	the omission of one or more observations – OPR-OMIT

- sz’  = [s2(      (_X_T  ___X_   )-1           )]1/2

- (****i)

- z’
- _b_  

- z’T
- _b_  

- (****i)

- (****i)

- (****i)

## OPR Statistic - Theory

- Weights on existing observations already determined
- Weights on potential observations must be determined using same guiding principles

- __

- Weights on existing observations and prior

## OPR Statistic - Calculation

- OBSOMIT STEPS:
- Set weight(s) for relevant observation(s) to zero
- Sensitivity matrix _**X**_ does not need to be modified
- Calculate sz
- OBSADD STEPS:
- Calculate sensitivities for potential observations and append these to _X_
- Construct weights for potential observations and append these to __Y,PRI
- Calculate sz

## Exercise 8.1d: OPR Statistic

- Use MODE=OPROMIT, OBSGROUPS=NO to analyze the individual omission of the existing head and flow observations and identify which of these observations are most important to the predictions.

## Exercise 8.1d – OPR Statistic Results

- Which observations rank as most important to the predictions?
- Why? Use:	dss – Table 7.5 (p. 148)	pss – Figure 8.8 (p. 198)	pcc – Information in Table 8.6 (p. 204)

- Figure 8.10a, p. 203

- OPR

## Exercise 8.1d – OPR Statistic Results

- Does analysis of the absolute increases in prediction standard deviation produce the same conclusions as did analysis of the opr statistics on the previous slide?

- Figure 8.10b, p. 203

- Change,
- in meters