# Presentation: 093647

## WP 4.1 Report of the Retrospective Working Group

*It is a simple but sometimes forgotten truth that the greatest enemy to present joy and high hopes is the cultivation of retrospective bitterness.* Robert G. Menzies (Australian Prime Minister 1930-1941&1949-1966)

## Retro WG Participants

- Special thanks to ICES WGMG

## Retro

- What is it?
- How is it measured?
- Why is it important?
- What causes it?
- Can we fix it?

## Retro Defined

*The retrospective problem is a systematic inconsistency among a series of estimates of population size, or related assessment variables, based on increasing periods of data.* (Mohn 1999)
- Historical vs Within Model
- Not just VPA

## Don’t Always Know It When You See It

## A Strong Retrospective

## Mohn rho

- 25 years in assessment
- 7 terms in rho calculation

## Woods Hole rho

- 25 years in assessment
- 300 (=24+23+22+...+1) terms in rho calculation

## rho Increases with Bias

## Why are retros important?

- Projections too high
- Catch advice high
- Causes F > Fref
- Feedback loop

## Causes of Retros

- Change in Catch (missing or extra)
- Change in M
- Change in Survey q
- Closed Areas (sessile)

## Missing Catch

## Change in M

## Change in Survey q

## Closed Area

## Retro by Chance Unlikely

## Can a Retro be Fixed?

- Yes
- Can remove retro pattern in many ways
- No
- Do not necessarily get closer to the truth
- Need to known timing and source to get it right
- Timing can sometimes be identified
- Source cannot be identified

## Local Influence Surface

- Cadigan and Farrell (2002, 2005)
- Use rho – no good
- Use other metric, can sometimes identify timing

## LIS with rho f(npeels)

## Split Surveys

- GB yt “solution” to retro
- Removes retro, but not necessarily to truth

## Change Catch (seen previously)

## Change Catch Split Surveys

## M Change (seen previously)

## M Change Split Surveys

## Change q (seen previously)

## Change q Split Surveys

## Source: Change q, Fix: Change M

## Retro Fix Resids

## Alternative “Fixes”

- Adjust NAA in Projections
- Assumes retro will persist
- Requires MSE to determine how to adjust
- Provide Alternative States of Nature Advice

## Conclusions

- 1. Retrospective pattern is an indication something is inconsistent (data and/or model).
- 2. Lack of a retrospective pattern does not mean all is well. Based on simulations, data or model inconsistency does not always produce a retrospective pattern. Retrospective patterning is just one diagnostic to be considered when conducting stock assessments.
- 3. Simulated retrospective patterns can be caused by time trending changes in biological characteristics, catch, survey catchability, or spatial concentration of the population. Multiple sources may occur in assessments.
- 4. The source(s) of the retrospective pattern can be anywhere in the time series. Some methods were presented to identify when the change took place (moving window, q surface, mean square residual LIS).
- 5. The true source(s) of a retrospective pattern have not been identified using current methods. Knowledge of events in the fishery or biological information may help identify probable sources.

## Conclusions (cont)

- 6. Interventions (correlated errors) are more likely to cause retrospective patterns than random noise.
- 7. Splitting surveys, changing M, or changing catch may reduce the retrospective pattern, but do not necessarily produce an assessment closer to the truth, although the other diagnostics for the new assessment may be fine.
- 8. The retrospective statistic, rho, may be a useful measure of the amount of retrospective pattern. A strong retrospective pattern can be defined by the degree of overlap between confidence intervals from different terminal years.
- 9. Local influence surface analysis using rho is not useful for diagnosing the timing or source of retrospective patterns.
- 10. In many stocks, strong retrospective patterns typically persist.

## Recommendations

- 1.	Always check for the presence of a retrospective pattern.
- 2.	If a model shows a retrospective pattern, then consider alternative models or model assumptions.
- 3.	Develop objective and consistent criteria for the acceptance of assessments with retrospective patterns.
- 4.	A strong retrospective pattern is grounds to reject the assessment model as an indication of stock status or the basis for management advice.
- 5.	When a moderate retrospective pattern is encountered: (not an exhaustive list)
- 	a.	Consider alternative states of nature approach to advice.
- 	b.	Investigate the performance of alternative methods for retrospective adjustments through management strategy evaluations.
- 6.	Use biological and fishery hypotheses and auxilliary information as a basis for adjustments for retrospective patterns.
- 7.	Consider use of survey swept area numbers instead of mean catch per tow in assessment models.
- 8.	The presence and implications of a retrospective pattern as a source of uncertainty in the assessment should be clearly communicated to managers.