# Presentation: 190297

## Calibration Guidelines

**1.** Start simple, add complexity carefully
**2.** Use a broad range of information
**3.** Be well-posed & be comprehensive
**4.** Include diverse observation data for ‘best fit’
**5.**** **Use prior information carefully
**6**. Assign weights that reflect ‘observation’ error
- 7. **Encourage convergence by making the model more accurate**** **
- 8. Consider alternative models

**9.****	**Evaluate model fit** **
**10.****	**Evaluate optimal parameter values
**11.**	Identify new data to improve parameter estimates
**12.**	Identify new data to improve predictions
**13.****	**Use deterministic methods
**14.**** **Use statistical methods

**Model development**

**Model testing**

**Potential new data**

**Prediction uncertainty**

## Model DevelopmentGuideline 7: Encourage convergence of the regression by making the model more accurate 

## Learn from Failed Regressions! 

- Nonlinear regression can be difficult
- Even when composite scaled sensitivities and correlation coefficients indicate the data provide sufficient information to estimate the defined parameters, nonlinear regression may not converge.
- Substantial insight about the observations, potential model inaccuracies, and model fit can be obtained from values calculated in failed regressions:
- dimensionless scaled sensitivities
- composite scaled sensitivities
- correlation coefficients
- influence statistics
- weighted and unweighted residuals
- parameter updates during the regression.

## Make the model more accurate 

- Working to make the model represent the system more accurately obviously is beneficial to model development, and generally also improves the behavior of the regression.
- Use the information from failed regressions, or regressions that converge to unrealistic parameter values, to guide changes.
- The most advantageous modifications for improving model accuracy are usually:
- Modify the parameter definition
- Modify other aspects of model construction, such as its physical features or the processes it simulates
- Estimate fewer parameters
- Add observations to the regression
- Scrutinize existing observations for error in interpretation

## The Main Problems that Plague Convergence...

- Insensitivity
- Nonlinearity
- Inconsistencies

## ...and Possible Solutions to Consider

- Insensitivity
- Specify parameters with small css
- Combine existing parameters
- Redesign the parameterization
- Creatively use system information (Guideline 2)
- Nonlinearity
- Evaluate results from intermediate iterations
- Evaluate large weighted residuals, observations omitted because simulated equivalents could not be obtained, whether parameter values are realistic
- If forward model nonlinearities are a suspected cause of nonconvergence, consider using a linear approximation
- Inconsistencies
- Check parameter representation, dominant observations
- Evaluate observations & prior, and their simulated equivalents

## Calibration Guidelines

**1.** Start simple, add complexity carefully
**2.** Use a broad range of information
**3.** Be well-posed & be comprehensive
**4.** Include diverse observation data for ‘best fit’
**5.**** **Use prior information carefully
**6**. Assign weights that reflect ‘observation’ error
- 7. Encourage convergence by making the model more accurate** **
- 8. **Consider alternative models (MMA)**

**9.****	**Evaluate model fit** **
**10.****	**Evaluate optimal parameter values
**11.**	Identify new data to improve parameter estimates
**12.**	Identify new data to improve predictions
**13.****	**Use deterministic methods
**14.**** **Use statistical methods

**Model development**

**Model testing**

**Potential new data**

**Prediction uncertainty**

## Guideline 8: Consider alternative modelsbook p. 308-314

- Developing alternative models
- Deterministic methods
- Alternative conceptual models about depositional environment
- Alternative theories about rainfall distribution and(or) infiltration dynamics
- Stochastic methods
- Alternative realizations of gravel/sand/clay distribution developed using indicator kriging
- Combined methods
- Discard some alternative realizations based on deterministic depositional theories
- Generate stochastic variations within a deterministically derived hydrogeologic framework

## Guideline 8: Consider alternative models

- Better models have three attributes:
- Better fit (But not too good!)
- Weighted residuals that are more randomly distributed
- More realistic optimal parameter values

- This graph shows model discrimination criteria for 5 models of the Maggia Valley, southern Switzerland.
- SSWR: Sum of squared, weighted residuals.
- AICc, BIC: Model discrimination criteria.

- Foglia, in press, GW. Book, p. 311.