# Presentation: 329906

## ICESat Ground System Element:NSIDC DAAC

**Melinda Marquis**
- Mission Operations Working Group Meeting
- January 30, 2002

## Overview

**System Architecture:  Distribution of Data by ECS**
**Current Status and Plans to meet Launch Readiness (Jan. 17, 03)**
- Milestones completed
- Future Milestones
**Testing**
- Ingest of data
- Metadata
- EDG (EOS Data Gateway)
**Documentation**
- DIFs
- Guide Documents
- Web Pages
- Documentation: Operations Procedures
**Staffing and Training**
- ECS
- Ops
**IT Security**
**rSCF at NSIDC**
**Schedules**

## System Architecture:  Distribution of Data by ECS

**EOS Data Gateway (EDG)**

**User**

**Data Management**
** ****Subsystem**

**Data Server Subsystem**

- Data Archive
- (Powderhorn)

- Science Data
-  Server Database

- Search and Order

- Granules Distributed

**Notes:**
EDG
-user places order
Order goes to Data Management Subsystem (DMS)
	DMS has lots of metadata, e.g. valids, etc., about collection level:
		temporal, spatial
	DMS includes the V0 to ECS Gateway:
		takes info from user/EDG and translates it into language that the Data Server Subsystem (DSS) can 			understand

Data Server Subsystem (DSS)
-the Brain
-comprises multiple machines/computers/servers and software systems
-stores data (contains the data archive: Powderhorn, < 300 terabytes, current medium is D3 tapes, soon to be replaced by IBM 9940 tapes)
-DSS holds metadata on each granule – in the Science Data Server Database
-DSS communicates with DMS, which goes back to EDG/user, informing:
EX: These granules match your request.  Which do you want to order?
Then user fills out order form, e.g., names, contact info., affiliation, distribution method (ftp push/pull, or 8 mm tape)
User receives 2 emails:
1: Your order has been received and you’ll be notified shortly of the ftp site, for example
2. Ftp site address

	INGEST IS AN SGI ORIGIN

## Milestones

| Task | Date |
| --- | --- |
| Draft DIFs (metadata documents) | Completed Feb. 01 |
| ICD between NSIDC and ISIPS | Baselined Dec. 01 |
| OA between NSIDC and ISIPS | Completed Jan. 02 |
| Interface Control Tests
(ICESat Ground System Tests) | IGST 4(July 01)
IGST 6 (July 01)
IGST 7 (Sept.-Oct. 01) |
| Successful MOSS 4 and 5, discovered and fixed problems | MOSS 4  Aug. 6-10, 01
MOSS 5  Oct. 1-5, 01 |
| Tested procedure for exchanging L0 replacement data with ISIPS (when either can’t easily retrieve it from EDOS) | Dec. 01 |
| Installed metadata files (ESDT descriptor files) and ingested test data for most data types | Throughout 2001 |
| Drafted two largest guide docs | Altimetry: March 01
Altmosphere: Fall 01 |
| Final edits of DIFs and guide docs, draft and edit GLA16 guide doc | By Oct. 15, 02 |

## Testing

- Mission Operations Science System (MOSS) Test 5

**Data Types Ingested from ISIPS:**
**Yet to be tested:**
- GLA03, 16
- ANCILLARY FILES:
- GLA ANC08
- GLA ANC09
- GLA ANC24
- GLA ANC25
- GLA ANC20

- GLA ANC22
- GLA ANC 33
-  GLA ANC 34
-   GLA ANC 37
-    GLA ANC 39

- GLA01, 02, 04-15

**Notes:**
Q: Doug, When will we test GLA03 and 16?

**Data Types Ingested**** ****	****Ingested    PDS      PDR**
-       GLA0CHT5  (#50)       10             13          10
-       GLA0SST1   (#24) 	     8             13            8
-       GLA0LST1   (#25) 	    11            13          11
-       GLA0LST2   (#55)        21            23          21
-       GLA0DATA  (#32)         9           13           10
-       GLA0CTDP   (#33)
-       GLA0MDP1  (#27)
-       GLA0MDP2  (#28)
-       GLA0EVNT  (#34)
-       GLA0MEM   (#35)
-       GLA0TBLE   (#36)
-       GLA0ETAL   (#37)
-       GLA0DTYP   (#28)
-       GLA0BORE   (#29)
-       GLA0PRAP    (#1984)    10          12          10
-       GLA0GPS       (#1088)    11          12          11

**Data Types Ingested ****	****Ingested****	****PDS     PDR**
** **     GLA0ADDL  (#12)	    11	13            11
-       GLA0ADDS  (#13)	    11	13            11
-       GLA0ADEM (#14)	      9	13              9
-       GLA0PCSP    (#15)	    11        	 13           11
-       GLA0PCEP    (#16)       12   	 13           12
-       GLA0CDSP   (#17) 	    12	13            12
-       GLA0CDEP   (#18)	    13	13            13
-       GLA0ASP      (#19)	      7	13              7
-       GLA0LPDP   (#26)        11	13            11
-       GLA0CHP     (#49)	     11            13            11
-       GLA0CHT1   (#20) 	     12	13            12
-       GLA0CHT2   (#21) 	     11	13            11
-       GLAOCHT3  (#22)          8	13             8
-       GLA0CHT4   (#23)         10	13            10
-       GLA0CODE  (#31)

- MOSS 5 Test Results: L0 Data Ingested from EDOS

**Notes:**
Q: Doug, Why is PDS > Ingested (=PDR, define?)  ?
Why are others blank?  When will these blank ones be tested?

## Testing: Metadata

- Status of Earth Science Descriptor Files (ESDTs)
- Have Received and Tested
- Metadata for:
- GLA00-15
- ANC08, 09, 20, 22, 24, 25, 33, 34, 37

- Have **Not** Received
**Metadata** for:
- GLA ANC39

## Testing:  EDG

- New 3.4 beta version of EDG will be tested during MOSS 6.
- Test NOSE granule map display.

## Future Tests (All with firewall installed)

- MOSS 6  Week of Jan. 28, 02
- MOSS 7  Week of Feb. 25, 02
- IGST 6.4  March, 02
- IGST 4     April, 02
- IGST 6.5   July, 02
- IGST 7s     Mid May – late Oct., 02

**Notes:**
Add IGST 4 (Feb.) and IGST 6 (Feb. and Aug.)

## Documentation

- DIFs (metadata documents submitted to GCMD)
- One for each of the 17 products (L0-L3)
- Completed and reviewed by D. Hancock and A. Brenner
- Keywords yet to be finalized for ESDTs, DIFs, and Guide Doc 	keywords (valids)

## Documentation

**Guide Documents**
- One guide document for all altimetry products: GLA01, GLA05, GLA06, GLA12-15
- First draft completed and reviewed by D. Hancock and A. Brenner
- One guide document for all atmospheric products: GLA02, GLA07-11
- First draft completed
- Will transfer both to new template that emphasizes procedures for working with data
- Next draft available by end of Feb. for review

## Documentation

- One guide document for GLA16 (the composite product in HDF-EOS format)
- Product not yet fully defined
- Once GLA16 is defined, we’ll draft guide document and have it reviewed by GLAS scientists.

**Guide Documents **

## Documentation Schedule

- Second drafts of altimetry and atmospheric guide documents
- completed by end of Feb., and will be provided to GLAS
- scientists for review
- Need details of parameters from D. Hancock NLT Aug. 15,
- for final versions
- In-house review of altimetry and atmospheric guide documents
- and final review of 17 DIFs will begin Aug. 1 and end Sept. 30
- GLAS scientists’ final review Oct. 1-15.

## Documentation

- Web Site
- GLAS Data Products at NSIDC
- _[http://nsidc.org/daac/glas/](http://nsidc.org/daac/glas/)_
- Mission overview
- Description of Products
- Description of Instrument
- FAQs
- Links to Related Sites

## Documentation: Operations Procedures

- We are using 611 procedures as basis, with NSIDC-specific modifications in some but not all cases
- All procedures will be documented on the internal NSIDC ECS website by 6/02

## Staffing: GLAS Product Team at NSIDC

| Melinda Marquis | Team Lead |
| --- | --- |
| Ruth Duerr | Data Operations Supervisor |
| Renea Ericson | Raytheon On-Site Manager |
| Siri Jodha Singh Khalsa | Site Liaison to ECS Science Office |
| Ted Scambos | Scientist |
| Anne Nolin | Scientific Consultant |
| Donna Scott | User Services Representative |
| Julie Soja | Valids and EOS Data Gateway Administrator |
| Jason Wolfe | Technical Writer |
| John Maurer | Data Operations Technician |
| Terry Haran | Scientific Programmer and Data Specialist |
| Cathy Fowler | Database Administrator |
| Doug Fowler | IV&V Test Engineer |
| Rick Pitre | ECS SSI&T Engineer |
| Michael Gehmeyr | ECS Configuration Management |

## Staffing and Training

- ECS
- ECS version 6A.05
- Training (Week of Jan. 21, 02)
- Installed (Feb., 02)

**Notes:**
ECS: EOSDIS Core System

Check with Renea

## Staffing and Training 

**Notes:**
70% = 50% <- full training
	  20% <- experience

Prod. Ops. Only 80% b/c 4 folks at 100%, minus 1 FTE

Review all #s with Ruth!

## IT Security: Audit Results

- ESDIS Security Audit (Oct., 02)
- NSIDC received excellent score
- Vulnerability rating of 0.06
- 0.25 is acceptable; the lower the score, the better
- Only 1 DAAC received score better than NSIDC’s
- Vulnerability ratio = _high vulnerability_
-                                      Total # of systems
- NSIDC: 	_4_
- 			  	70

**Notes:**
ASK TODD:  WHAT’S THE WORST SCORE POSSIBLE?  >1?  Yes, infinite, theoretically.
CAN THERE BE >1 VULNERABILITY/SYSTEM ? Yes

## IT Security: Improvements After Audit 

- Audit performed in Oct. 2001
- By Jan. 8, 2 of 4 vulnerabilities eradicated
- Now, vulnerability rating is 2/70 = 0.03
- Working to address remaining 2 vulnerabilities

## It Security:  Audit Results

- -- At time of Audit, Oct. 01

- 0.25, acceptable

- 0.06, **Excellent NSIDC**

- Unacceptable

- 8

## IT Security

- Excerpts from audit, by Titan Systems Corp.:
- “I also want to compliment you on your scanning procedure that you have implemented, such efforts makes [sic] our job easier and allows us to focus our time on sites that need it.
- You are setting a great example for other DAACs to follow and proves [sic] that a DAAC can operate in a secure fashion.”
- Ronald W. McClellan, Principal INFOSEC Engineer

## IT Security: Firewall

**ECS Router**

- UCB Campus & Public Internet

- M&O Network

**Notes:**
Installed Jan. 11-16
NASA-approved firewall for DAAC configuration
-which network services are allowed.
EMSN:  EDOS, ISIPS, AMSR-E SIPS, MODAPS.
Private subnet (VLAN = Virtual Local Area Network).
CHECK SLIDE WITH TODD.

## IT Security

- Documents baselined
- NSIDC DAAC Information Technology (IT) Major Application System Security Plan (Dec., 00)
- NSIDC DAAC Information Technology (IT) Risk Management Plan (Dec., 00)
- NSIDC DAAC Information Technology (IT) Contingency  Plan (Dec., 00)

## IT Security: Other Security Measures taken at NSIDC

- Daily Monitoring of network security logs
- All relevant systems have TCPwrappers installed
- Tripwire installed
- SSH: Secure Shell in use in lieu of telnet
- Install patches to OS and other COTS packages, as needed
- Support ESDIS audits every 6 months

**Notes:**
Tripwire  -- looks for files that have been modified 
SSH – telnet replacement

## rSCF at NSIDC

- Procurement completed
- Delivery and installation of system H/W & S/W in early Feb., 02
- Install SCF interface S/W for rSCF mid to late Feb., 02
- (Will be contacting John DiMarzio.)
- Install SCF visualization S/W
- Begin test and operator training March, 02

**Notes:**
Justification/Value:  
The acquisition of this workstation and providing NSIDC staff a working knowledge of the SCF’s software is critical to the success of the GLAS data management mission at NSIDC.  First, NSIDC is ‘the’ snow and ice DAAC, and GLAS data will be extremely valuable to cryospheric researchers.  NSIDC often has visiting scientists, and it is reasonable to expect some of them to wish to use the SCF’s software for visualizing GLAS data when they are visiting us.  Second, it is reasonable to expect the GLAS science team to distribute the SCF software to cryospheric scientists.   The SCF has the responsibility to distribute and support their software.  If users contact NSIDC USO for help with the SCF software, the USO representative will refer the user to the SCF.  

Additionally, an NSIDC staff scientist is interested in using the workstation, from both a science (research) stand-point and as someone involved in fielding questions about the data products.  This is critical to our providing a good level of service to our users.

Once we have an HP work station, we shall evaluate for the feasibility of porting the software to another system.  We will compose a report on our findings.  If we find that it is feasible to port the software and thus distribute it to users who request it, then we will seek additional funding to support such activity.

QUESTION:  WILL WE USE RSCF SUBSETTER TO SUBSET DATA FOR OUR USERS?

NSIDC is purchasing a HP work station, the only platform upon which runs the GLAS SCF data visualization software. 
The software allows access to data and ability to visualize parameters of a complete suite of GLAS products (except L0).  
Software allows for subsetting (temporal and spatial), geolocation, browse and Q/A data.  
It shows ground tracks, with the option to plot them over a DEM on a regional map. For altimeter products, it shows height profiles of the track selected: x-y plots for corrections, waveforms, waveform parameters, and DEM profiles. 

## Schedules

**Notes:**
SORR
Initial dry run June 5, 2002 (telecon)
Final dry run July 10, 02
Real thing Aug. 21, 2002

## Risk Areas and Mitigating Factors

- No known risk areas

## Any other issues that may impact mission readiness

- No known issues

## Technical Issues and Concerns

- No known technical issues or concerns

## BACKUP SLIDES FOLLOW

## GLAS Data Products

## Comprehensive Documenation Status

**NSIDC ECS & I-SIPS ICD**    	Baselined in Dec. 01
**NSIDC ECS & I-SIPS Volume 0** 	Baselined Dec. 23, 98 (Revision E released Nov. 14, 01.)
**NSIDC Operations Agreement NSIDC-ISIPS**    Completed.  Hancock et al. will sign when Marquis is at GSFC for MOWG Jan. 30, 02
**NSIDC Operations Agreement NSIDC-EDOS**    I've asked Bob Lutz to confirm that a separate OA between NSIDC and EDOS  for ICESat/GLAS data is not needed.  He has agreed, but wants to confirm this with someone else. Such an additional OA seems unnecessary, in light of the appendix in the EGS-EDOS ICD.  Also, there is an OA between EDOS and the ECS SDPS (510-OA-EDOS/SDPS).
**NSIDC Operations Procedures Document**     Most of the components are complete now.  Others will be complete by Feb. 15.  All procedures will be documented on an internal NSIDC ECS web site by June. A PowerPoint Slide that we used for the Aqua Interim ORR Jan. 9 gives more details.
**NSIDC Data Set Guide Documents**    The altimetry guide document was drafted and reviewed by  Hancock and Brenner in May, 01 and July 01, respectively.  The atmospheric products guide document has been drafted by not yet reviewed by GLAS scientists. Both of these guide documents will be further edited and sent to GLAS scientists for another review by Feb. 28, 02. David Hancock is still changing some of the product parameters, and we will need these nailed down no later than Aug. 15. In-house (NSIDC) review of altimetry and atmospheric guide documents will begin Aug. 1 and end Sept. 30.  GLAS scientists’ final review will be Oct. 1-15.  The GLA16 product (the composite product in HDF-EOS format) is not yet defined. Once defined, we’ll draft guide document and have it reviewed by GLAS scientists.
**NSIDC Data Interchange Format Documents**    These were drafted and reviewed by Hancock Feb. 01.  Final in-house review of these 17 DIFs (after Hancock nails down final parameters) will be Aug. 1-Sept. 30, and final review by GLAS scientists Oct. 1-15.

## GLAS Data Flow

## Slide 35

| Data Product | Description/Parameters | Short Name, Processing
Level | Granule/File Size |
| --- | --- | --- | --- |
| Global 
Altimetry 
Data | The level 1A altimetry data (GLA01) will be at the full 40 per second resolution.  Data granules will contain approximately 23 minutes (1/4 orbit) of data and will include pulse travel times, the transmitted and received waveform, transmitted and received energy, echo time offsets, and instrument filter and threshold values.  The data will be geolocated to the sub-satellite location. These data will serve as input to determining surface elevations, slopes and relief characteristics.  This product is used as input to create GLA05. | GLA01, L1A | 34 MB, ~1/4 orbit |
| Global 
Atmosphere 
Data | The level 1A atmospheric data (GLA02) will be at full instrument resolution.  Data granules will contain approximately 190 minutes (2 orbits) of data and will include normalized relative backscatter, photon counter, cloud digitizer, and timing, data which will serve as input to higher level atmospheric products. The data will be geolocated to the sub-satellite location. This product is used to create GLA07. | GLA02, L1A | 703 MB,  2 orbits |
| Global 
Engineering Data | The level 1A global engineering data (GLA03) data granules contain approximately 190 minutes (2 orbits) of satellite housekeeping data. These are used to calibrate values for GLA01 and GLA02.  | GLA03, L1A | 13 MB,  2 orbits
 |
| Global Laser Pointing Data | The level 1A global laser pointing data (GLA04) provide attitude data from the spacecraft star tracker, instrument star tracker, gyro, and laser reference system, and other spacecraft attitude data required to produce precision attitude data. These data granules will contain approximately 3 hours (2 orbits)  in length and are used for calculation of precise laser pointing.  | GLA04, L1A | 500 MB, 2 hours |
| Global Waveform-
based Range 
Corrections Data | The level 1B waveform parameterization data will contain waveform-based range corrections and surface characteristics (GLA05) at the full 40 per second resolution.  Data granules will contain approximately 23 minutes (1/4 orbit) of data and will include the output from the waveform characterization procedure and other parameters required to calculate surface slope and relief characteristics.  These data will be geolocated to the center of the laser footprint. This is used as input for GLA06, and GLA12-15.  | GLA05, L1B | 21 MB,  ~1/4 orbit |

## Slide 36

| Global Elevation
Data | The level 1B elevation data (GLA06) will be at the full 40 per second resolution.  Data granules will contain approximately 23 minutes (1/4 orbit) and  will include the surface elevation, surface roughness assuming no slope, surface slope assuming no roughness and geodetic and atmospheric corrections for the range.  These data will be geolocated to the center of the laser footprint. This product will be used with GLA05 to create the level 2 altimeter products (GLA12-15).   | GLA06, L1B | 9 MB,  ~1/4 orbit |
| --- | --- | --- | --- |
| Global 
Backscatter
Data | The level 1B global backscatter data (GLA07) will be provided at full instrument resolution. The product will include full 532 nm (41 to -1km) and 1064 nm (20 to -1 km) calibrated attenuated backscatter profiles at 5 times per second, and from 10 to -1 km, at 40 times per second for both channels.  Also included will be calibration coefficient values and molecular backscatter profiles at once per second. Data granules will contain approximately 190 minutes (2 orbits) of data. | GLA07, L1B | 828 MB,  2 orbits |
| Global Planetary 
Boundary Layer 
and Elevated 
Aerosol Layer Heights | The level 2 planetary boundary layer and elevated aerosol layer height data (GLA08) will be provided at once per 4 seconds. Planetary boundary layer (PBL) height data will be provided at once per 4 seconds and at 5 times per second. Elevated aerosol layer height data will consist of top and bottom heights for up to 5 aerosol layers below 20 km at once per 4 seconds, and top and bottom heights for up to 3 aerosol layers above 20 km at once per 20 seconds. Data granules will contain approximately 23 hours (14 orbits) of data.  | GLA08, L2 | 2 MB, 14 orbits |
| Global Cloud
Heights for Multi-layer Clouds | The level 2 cloud layer height data (GLA09) will be provided at once per 4 seconds. Cloud layer height data will consist of top and bottom heights for up to 10 layers below 20 km at once per 4 seconds, once per second, 5 times per second, and 40 times per second (below 4 km only). Ground heights will also be provided at each resolution. Data granules will contain approximately 23 hours (14 orbits) of data.  | GLA09, L2 | 81 MB, 14 orbits |
| Global Aerosol 
Vertical Structure
Data | The level 2 aerosol vertical structure data (GLA10) will be provided at once per 4 seconds and will contain cloud and aerosol backscatter and extinction cross section profiles.  Cloud data will be provided at once per second and aerosol data will be provided at once per 4 seconds. Data granules will contain approximately 23 hours (14 orbits) of data.  | GLA10, L2 | 301 MB, 14 orbits |
| Global Thin Cloud –Aerosol Optical Depths Data | The level 2 thin cloud/aerosol data (GLA11) will be provided at once per 4 seconds and will contain optical depths for clouds for 
up to 10 layers, the planetary boundary layer, and aerosols for up to 8 layers.  Cloud data will be provided at once per second and aerosol data will be provided at once per 4 seconds.  Data granules will contain approximately 23 hours (14 orbits) of data.  | GLA11, L2 | 12 MB, 14 orbits |

## Slide 37

| Antarctic and 
Greenland Ice
Sheet Altimetry
Data | The level 2 ice sheet altimetry product  (GLA12) contains the ice sheet elevation and elevation distribution calculated from algorithms fine-tuned for ice sheet returns. It will also include the laser footprint geolocation, reflectance, and the geodetic, instrument, and atmospheric corrections used to correct the range measurements. Parameters will be at the full 40 per second resolution, but only for locations that fall within the ICESat ice sheet mask.  Data granules will contain approximately 23 hours (14 orbits) of data over the ice sheets.  | GLA12, L2 | 49 MB,  14 orbits |
| --- | --- | --- | --- |
| Sea Ice Altimetry
Data | The level 2 sea ice altimetry product  (GLA13) contains the sea ice freeboard and sea ice roughness calculated from algorithms fine-tuned for sea ice returns.  It will also include the laser footprint geolocation, reflectance, and the geodetic, instrument, and atmospheric corrections used to correct the range measurements. Parameters will be at the full 40 per second resolution, but only for locations that fall within the ICESat sea ice mask.  Data granules will contain approximately 23 hours (14 orbits) of data over the sea ice regions.  | GLA13, L2 | 22 MB, 14 orbits |
| Global Land 
Surface 
Altimetry Data | The level 2 land altimetry product  (GLA14) contains the land elevation and land elevation distribution calculated from algorithms fine-tuned for land returns.    It will also include the laser footprint geolocation, reflectance, and the geodetic, instrument, and atmospheric corrections used to correct the range measurements.  Parameters will be at the full 40  per second resolution, but only for locations that fall within the ICESat land mask.  Data granules will contain approximately 23 hours (14 orbits) of data over the land regions.  | GLA14, L2 | 273 MB, 14 orbits |
| Ocean Altimetry
Data | The level 2 ocean altimetry product  (GLA15) contains the ocean elevation and small-scale roughness calculated from algorithms fine-tuned for ocean returns.    It will also include the laser footprint geolocation, reflectance, and the geodetic, instrument, and atmospheric corrections used to correct the range measurements.  Parameters will be at the full 40 per second resolution, but only for locations that fall within the ICESat ocean mask.  Data granules will contain approximately 23 hours (14 orbits) of data over the oceans.  | GLA15, L2 | 240 MB, 14 orbits |
| Global Elevation
and Atmosphere
HDF-EOS Data | The level 3 elevation and atmosphere data (GLA16) will contain derived geophysical parameters. Elevation data will include corrected elevations, reflectivity, and surface roughness for ice sheets, sea ice, land, and oceans at 40 Hz, 5 Hz, and 1 Hz. Atmospheric data will include cloud and aerosol layer heights, aerosol optical depth, backscatter and extinction values, and polar stratospheric cloud top and bottom heights. Data granules will contain 23 hours (14 orbits) of data in HDF-EOS format.  | GLA16, L3 | 117 MB, 14 orbits |