# Presentation: 294013

## A study of marine primary productivity models, with an ocean color biasPrimary Productivity Round Robin, the III and IV(PPARR3 and PPARR4)

- Mary-Elena Carr1, Marjorie Friedrichs2,
- Richard Barber3, Marjorie Schmeltz1,
- and the PPARR3 Group (~30 investigators)
- 1 Jet Propulsion Laboratory
- 2 Old Dominion University
- 3Duke University

*We thank the wonderful PPARR3 participants for their hard work and NASA’s *
*Oceanography Program for ongoing support. *
*MEC acknowledges the National Science Foundation for support.*

## Motivation and History

- Primary productivity, or carbon fixation through marine photosynthesis, is an important component of the global carbon cycle. Remote sensing enables quasi-synoptic global observation of near-surface chlorophyll concentrations.
- There exist several models of varying complexity which estimate primary productivity from ocean-color based chlorophyll or irradiance values, hereafter PP models.
- In PPARR2, a blind intercomparison to *in situ* data, the best performing algorithms were within a factor of two. The equatorial Pacific and Southern Ocean data presented larger offset.***Our goal is to provide a framework to systematically compare models which estimate primary productivity from chlorophyll measured by ocean color.****** ***

## Outline

- Motivation
- How do we compare algorithms
- PPARR3 Part 3
- Approach
- Statistical measures of skill
- Conclusions
- What drives the model error
- Conclusions
- Comparison with PPARR2
- Conclusions
- PPARR4

## PPARR3: Our approach

**PART 1.** Annual cycle (6 months of 1998 and December 1999). *Model output intercomparison. *
*In press DSRII, third special volume on JGOFS-SMP.*
**PART 2.** Sensitivity analysis exploring biomass determination and parameterization of light utilization and photo-adaptive physiology. *Sensitivity to input variables.*
*In press DSRII, third special volume on JGOFS-SMP.*
**PART 3.** Comparison to *in situ* 14C uptake (ClimPP: ~ 1000 tropical Pacific stations). *Ground-truth comparison.*
*In prep, and Friedrichs et al 2006 AGU Ocean Sciences Poster. *

## Part 3: Comparison with 14C uptake measurementsFriedrichs, Carr, Barber, Scardi, Schmeltz, & PPARR3ers 

-  The offsets between in situ 14C uptake rates and estimates of integrated primary productivity obtained from PP models were particularly large in the equatorial Pacific (PPARR2, Campbell et al. 2001)
-  Based on a large quality-controlled database (ClimPP): ~1000 14C measurements spanning more than a decade (1983-1996) in the tropical Pacific (Barber).

## Participating Models

| 1 | Friedrichs (sqrt(chl)) | WIDI |
| --- | --- | --- |
| 2 | Carr (HoYoZe) | WIDI |
| 3 | Carr (HoYoRy) | WIDI |
| 4 | Carr (Ryan) | WIDI |
| 5 | Scardi | WIDI |
| 6 | Carr (VGPM) | WIDI |
| 7 | O’Malley & Behrenfeld (VGPM Eppley) | WIDI |
| 8 | Ishizaka & Kameda | WIDI |
| 9 | Ciotti | WIDI |
| 10 | Asanuma | WIDR |
| 11 | Marra | WIDR |

| 12 | Smyth (LUT Morel) | WRDR |
| --- | --- | --- |
| 13 | Smyth | WRDR |
| 14 | Melin & Hoepffner | WRDR |
| 15 | Waters & Bidigare | WRDR |
| 16 | Antoine & Morel | WRDR |

| 17 | Christian | GCM |
| --- | --- | --- |
| 18 | Moore & Doney | GCM |
| 19 | Dunne | GCM |
| 20 | Gregg | GCM |
| 21 | Gregg | GCM |
| 22 | Buitenhuis & LeQuere | GCM |
| 23 | Chai | GCM |

***Participating Models***

*WIDI=wavelength and depth integrated; WIDR= wavelength integrated and depth resolved; WRDR= wavelength and depth resolved; GCM= global circulation models*

## Approach-1

-   We distribute the input files and the participants estimate depth-integrated PP and give us the output, which we then compare.
-   For Part 3 (and for PPARR4), instead of using chlorophyll measured from ocean color, we use the surface chlorophyll from *in situ* data.

## Approach-2

- PPARR2 (Campbell et al. 2001) concluded that the most useful method to compare PP models is to compute the total* RMS* error (*RMS_T*) in log space:
- 	*PPm(i) *= modeled PP values
- 	*PPd(i)*  = observed PP values
- Total *RMS* is composed of two components
-  bias *RMS* (*RMS_B*): compares model mean and data mean
-  centered pattern *RMS* (*RMS_V*): compares variability of model and that of data

## Is model error due to bias or variability?

- For each of the participating models, total *RMS* error, *RMS_B*, and *RMS_V* are computed for the entire data set (top) and for post-1990 (bottom).
- Model performance improves after 1990.

-   1983-1996:
-  *RMS_V* ~ 0.25
-  *RMS_B* +/-0.2
-  All but 5 *RMS_B* >0
-  1990-1996
-  *RMS_V* ~0.2
-  *RMS_B* +/-0.3
-  All but 5 *RMS_B* ~<0
-  Solid concentric semi-circles represent *RMS_T *isolines of 0.2 and 0.3.
-  Model *RMS_V* tend to cluster.

*Blue = WIDI;** **Dark blue = VGPM variants;** **Red = WIDR; *
*Green = WRDR;** **Pink = GCM-based** (Model 23 only run for 1990s.)*

## Statistical comparison with observations

- Taylor diagrams present *RMS_V*, correlation, and standard deviation for modeled variables (*log(PP)* here) in a single plot*.*
-  Standard deviation = distance from the origin
-  Correlation between observations and model = azimuth angle
-  *RMS_V*= distance between observations and model
- Solid black diamond and dotted line represents the standard deviation of the data.
- Model RMS_V is lower in the latter part of the record, correlation increases to >0.8, but standard deviation is the same or less.
- Dashed lines are isolines of *RMS_V*=0.25 and *RMS_V*=0.15.

*Blue = WIDI;** **Dark blue = VGPM variants;** **Red = WIDR; *
*Green = WRDR;** **Pink = GCM-based** (Model 23 only run for 1990s.)*

## Model performance is independent of traditional groupings of model structure (i.e. depth/wavelength resolution, satellite-based algorithm vs. GCM-based). 

Almost all models have lower RMS_T and RMS_V for data collected after 1990, especially the satellite-based PP algorithms. (RMS_B changes less and interestingly goes from a positive mode to a negative mode.)

All satellite-based PP algorithms and all but one ecosystem model underestimate the observed variance of PP. Models #21, 22 (Gregg) most closely reproduce the observed variance for the entire time series.

WRDR present the lowest RMS_T value for the full data set: #14 (Melin & Hoepffner), while #16 (Antoine & Morel) and #13 (Smyth) for 1990-1996. That of the WIDI is slightly larger, such as #9 (Ciotti, VGPM variant) for the entire period or #3 (Carr, HoYo variant) for 1990-1996.

The simplest model (PP=chl1/2) (#1) does well in terms of RMS_T, however the variance is much too low. 

- Almost all models have lower *RMS_T* and *RMS_V* for data collected after 1990, especially the satellite-based PP algorithms. (*RMS_B* changes less and interestingly goes from a positive mode to a negative mode.)
- All satellite-based PP algorithms and all but one ecosystem model underestimate the observed variance of PP. Models #21, 22 (Gregg) most closely reproduce the observed variance for the entire time series.
- WRDR present the lowest *RMS_T* value for the full data set: #14 (Melin & Hoepffner), while #16 (Antoine & Morel) and #13 (Smyth) for 1990-1996. That of the WIDI is slightly larger, such as #9 (Ciotti, VGPM variant) for the entire period or #3 (Carr, HoYo variant) for 1990-1996.
- The simplest model (PP=chl1/2) (#1) does well in terms of *RMS_T*, however the variance is much too low.

***Conclusions-1***

## What is the main driver of model error?  (1) 

-  We estimate the correlation between the model error, D= log*PPd* – log*PPm,* and ten environmental variables at each space-time point:
-   1. depth of the chlorophyll maximum
-   (ZChlm )
-   2. mixed-layer depth (MLD)
-   3. distance from the equator (Lat)
-   4. sea surface temperature (SST)
-   5. surface chlorophyll (Chl0)
-   6. year day (YD)
-   7. photosynthetically-active radiation (PAR)
-   8. longitude (Lon)
-   9. year (Yr)
-  10. biomass normalized PP integrated
- over the euphotic zone (Pb)

- The high correlation between model error and Pb as well as between model error and year result from the models uniformly overestimating PP data for which the Pb is low. These low Pb observations are almost exclusively from the 1980s.

## What is main driver of model error?  (2)

- In an attempt to understand model error, D= log*PPd* – log*PPm, *we  plot it as a function of in situ Pb, mmol C  (mg Chl * day)-1 in the euphotic zone.
- For all models, the model error is maximum for Pb< 1. The models overestimate PP for low Pb.
- Note that the lowest Pb values occurred in the 1980s.
- Each plot shows error for each model as a function of Pb.

## Is low  Pb due to higher chlorophyll?

- Nope.
- Chlorophyll concentrations prior to 1990, when Pb is lowest, are not higher than later in the time series.

## Conclusions-2

- Correlation between year and model error is high: error is much smaller in the 1990s than in the 1980s.
- Pb is also much lower during the 1980s than during the 1990s, indicating either an actual change in the equatorial Pacific ecosystem or a change in sampling methodology.
- Correlation between Pb and model error is also high for all ocean color-based PP algorithms and most ecosystem models: models do poorly at low Pb.
- This may result from a limitation of the algorithms, which are primarily driven by chlorophyll or may reflect the fact that most of the models were developed using post-1990 data rather than pre-1989 data.
- Satellite-based PP model error is higher in the eastern Pacific and higher when surface chlorophyll is low and the deep-chlorophyll maximum is relatively deep.

## Comparison with model performance in PARR2

- 	PPARR2 used the JGOFS 1992 data collected on and off the equator at 140oW to compute the average *RMS_T* error for all participating models. An analogous calculation is performed using the PPARR3 models.

*Only models 2 through 16 are used in the comparison *
- Total *RMS* error of participating models in PPARR3 has significantly decreased from those of PPARR2, both on the equator and off the equator.

|  | # of models | Avg RMS
on equator | Avg RMS 
off equator |
| --- | --- | --- | --- |
| PPARR2 | 12 | 0.50 | 0.31 |
| PPARR3 * | 13 | 0.22 | 0.21 |

## Conclusions for PPARR3 Part 3

-   Model error is independent of whether or not the models resolve wavelength and/or depth, or whether satellite-based algorithms or ecosystem models are used.
-  Within the tropical Pacific, models perform significantly better in the 1990s than in the 1980s.
-  The 1980s are associated with low Pb values. Effort needs to be placed on improving the performance of PP models when Pb < 1.  We are collecting more data to fully evaluate the temporal evolution of Pb in the tropical Pacific.
-  The success of PPARR2 is illustrated by the decrease in mean *RMS* error.  The success of PPARR3 is evidenced by the continual modification and improvement of the participating models.

***Conclusions for PPARR3 Part 3***

## Where from here: PPARR4

- Recently funded.
- Solely based on ground-truth comparisons.
- Regions:
- JGOFS Process Studies:
- Arabian Sea
- North Atlantic
- Southern Ocean
- Revisit Equatorial Pacific: and whassup with the change in Pb?
- JGOFS Time Series
- Bermuda Atlantic Time Series
- Hawaii Ocean Time-series
- Global coastal database (compiled by MicheleScardi).
- Antarctic Peninsula, coastal and COLD! (compiled byHeidi Dierssen)

## PPARR4: Current Status

- We have initial invitation out to PPARR3 community
- About to send broader invitation out
- We welcome GCM or ocean color modelers that work in a small region.
- Have compiled all data needed for HOT and BATS distribution, now doing QC and data comparison.
- Anticipate sending it out in late May or early June