# Presentation: 203073

## PGE Delivery and Data Processing Plansfor theVersion 4.0 Software Release

**Steven Friedman**
**AIRS Science Processing**
**March 30, 2004**

## Topics

**Version 4.0 Schedule**
**Version 4.0 Data Processing Plans**
**Routine Data Processing at GSFC DAAC **
**Data Processing at AIRS TLSCF**
**Version 4.0 Reprocessing Considerations**

##  Strange Archeological Sites in the LA Area

## Version 4.0 Schedule – Key Objectives

**Version 4.0 to include enhanced validation coverage area**
**Extends coverage to land areas**
**Expands latitude range beyond ********40°**
**Feature Set Enhancements to include:**
**Level 1**
**AIRS IR****– Calibration enhancements - New gains, limits, etc.****– Geolocation,    Potential algorithmic upgrades (?)**
**Microwave****– Calibration enhancements,    Algorithmic upgrades (?)**
**Level 2 – *****the key ingredient to V4.0*****:**
**Further improvement beyond V3.0 and V3.5 baselines**
**Incorporates many performance enhancements**
**Includes better retrieval algorithm, handling of outliers, QA**
**Final feature set must be solidified by early summer!**

## V4.0 Timeline

## Version 4.0 Milestones

**V4.0 – Next Release to DAAC & NOAA**
**	**Prototyping Ends	6/01/04
- 	**Final Inputs to CCB****	****6/01/04**	*Last day to propose updates*
- 	**Development Period Ends****	****7/05/04**
- 	Initial Build – V4.0	7/06/04	*Anomalies / small changes*
- 	V4.0 Operational – Begin Validation DP	7/21/04	*Critical anomalies ONLY!*
- 	Delivery to DAAC & NOAA	8/31/04
- 	**V4.0 Operational – Begin Processing****	****9/30/04**
- 	V3.5 – Interim Release to NOAA	4/16/04	Improved L2, QA
- 	V3.7 – Internal JPL Benchmark	5/03/04	L3 Test, L2 In-process

**Please keep in mind:**
** ****All scheduled milestones are established to ensure that we deliver a quality V4.0 product on schedule.**
**There will always be new and better improvements, and there are other releases planned for the future.**

## Routine Data Processing at the GSFC DAACwith V4.0

**With V4.0 the GSFC DAAC will process:**
**Level 0 through Level 2 PGEs + Browse PGEs**
**Level 1B:  calibrated radiances, QA subsets, daily browse**
**Level 2:  retrieval standard,, retrieval support,****	****CC radiances & daily browse (retrieval and CC)**
**Level 3 PGE – 3 *****new***** standard products for V4.0**
**Daily, 8-day, and monthly****– additional products planned for later releases**
**Subsetted data sets**
**Considering developing new subsetted radiance data sets**

## Routine Data Processing at the GSFC DAACwith V4.0 (cont’d.)

**With V4.0 the GSFC DAAC will also process:**
**Match-Up PGE *****(products delivered to JPL ONLY!) ***
**Radiosonde Match-Up – supports trending analysis**
**Fixed-location* Match-Ups**
- Dedicated Validation Sites
- 5 special validation sets – ACAR, ACFT, Dobson, Dobson_P, Yoe
- *No Dynamic Match-Ups will be performed at the GSFC DAAC– a new fixed-location Match-Up is being developed:   ***fixed buoys***

## Data Processing at the AIRS TLSCFwith V4.0

**Beginning with V4.0 the JPL TLSCF will:**
**Stop baseline processing:**** – Level 0 through Level 1B + Match-Up**
**Continue to process Focus Days (nominally, 48-day intervals)****– starts with Level 0 producing the following products:**
- Level 1A:	all AIRS, AIRS VIS, and AMSU products
- Level 1B:	calibrated radiances, QA subsets, daily browse
- Level 2:	retrieval standard, CC radiances, retrieval 			support, & daily browse (retrieval and CC)
- Level 3	daily product
**Continue to process other data as necessary**
- Support continued software development & test
- Support validation campaigns
- We will order additional Level 0 and/or Level 1B as needed

## Data Ingest at the AIRS TLSCFwith V4.0

**Beginning with V4.0 the JPL TLSCF will:**
**Stop ingesting Level 0 from the GSFC DAAC on a daily basis **
**Continue ingesting Level 0 from the GSFC DAAC for**
- All Focus Days
- Special campaigns
**Continue ingesting products from the GSFC DAAC**
- All Level 2 standard retrieval products
- All Daily Browse Products – Level 1B and Level 2
- Radiosonde Match-Up Products
- Occasionally ingest 20 granules of all Levels 0,1 and 2 products(support spot checking)
**Begin ingesting Level 3 and fixed-site Match-Up Products**
**Continue ingesting selected granules from NOAA NRT process**

## Processing and Reprocessing  at the GSFC DAAC with V4.0

**GSFC DAAC will begin processing with V4.0 when the release becomes operational**
**All forward processing will be performed with V4.0**
**Reprocessing will commence providing coverage back to****August 31, 2002 and can be accomplished by:**
- Marching backwards towards August 31, 2002? 	– or –
- Marching forwards from August 31, 2002?
**Reprocessing includes rerunning all data from Level 0****– Performs Level 1A &1B, Level 2, Match-Up and Browse PGEs**
**For AIRS, allocated processing resources are 3x**
- 1x dedicated to forward processing
- 2x dedicated to reprocessing
*It will take approximately 11⁄4 years to complete reprocessing**(may be less time, given Level 2 performance improvements)*

## Reprocessing Considerations after V4.0

**After V4.0, reprocessing will not keep up with annual upgrades**
- Continuous improvement of PGEs, delivered to DAAC on an incremental basis – deliveries scheduled annually
- Our desire to produce the best possible product versus the DAAC’s desire to produce complete collections
**Something has to give... and processor resources are fixed**
- Either we hold back deliveries to pace reprocessing			– or –we consider a hybrid approach after V4.0
**Could we consider stabilizing Level 1A(?) and Level 1B(?) with V4.0 until the last reprocessing cycle at end of mission?**
- Allow table upgrades, but delay science algorithm changes
- Would dedicate processor resources to Level 2 reprocessing

## V3.0 Reprocessing Status at the GSFC DAAC

**Processing status as of March 22, 2004***

***Solid bars indicate unprocessed data.**

**Calendar Day of Year**

**2004**

**2003**

## Slide 14