Comprehensive Large Array-data Stewardship System (CLASS)

Archive and Access System Requirements

**Version 2.2**

**16 May 2005**

***Prepared by:***

**U.S. Department of Commerce**

**National Oceanic and Atmospheric Administration (NOAA)**

**National Environmental Satellite, Data, and Information Service
(NESDIS)**

Document Change Notice

+---+-------+---+-------+---------------------+---+---------------------+
| D |       | D |       | PROGRAM: **CLASS**  | P |                     |
| C |       | A |       |                     | A |                     |
| N |       | T |       |                     | G |                     |
| N |       | E |       |                     | E |                     |
| O |       | : |       |                     | N |                     |
| : |       |   |       |                     | O |                     |
|   |       |   |       |                     | : |                     |
|   |       |   |       |                     | * |                     |
|   |       |   |       |                     | * |                     |
|   |       |   |       |                     | 1 |                     |
|   |       |   |       |                     | o |                     |
|   |       |   |       |                     | f |                     |
|   |       |   |       |                     | 1 |                     |
|   |       |   |       |                     | * |                     |
|   |       |   |       |                     | * |                     |
+---+-------+---+-------+---------------------+---+---------------------+
| D |       |   |       |                     |   |                     |
| O |       |   |       |                     |   |                     |
| C |       |   |       |                     |   |                     |
| U |       |   |       |                     |   |                     |
| M |       |   |       |                     |   |                     |
| E |       |   |       |                     |   |                     |
| N |       |   |       |                     |   |                     |
| T |       |   |       |                     |   |                     |
| T |       |   |       |                     |   |                     |
| I |       |   |       |                     |   |                     |
| T |       |   |       |                     |   |                     |
| L |       |   |       |                     |   |                     |
| E |       |   |       |                     |   |                     |
| : |       |   |       |                     |   |                     |
|   |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| C |       |   |       |                     |   |                     |
| L |       |   |       |                     |   |                     |
| A |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| A |       |   |       |                     |   |                     |
| r |       |   |       |                     |   |                     |
| c |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| i |       |   |       |                     |   |                     |
| v |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| n |       |   |       |                     |   |                     |
| d |       |   |       |                     |   |                     |
| A |       |   |       |                     |   |                     |
| c |       |   |       |                     |   |                     |
| c |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| s |       |   |       |                     |   |                     |
| s |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| y |       |   |       |                     |   |                     |
| s |       |   |       |                     |   |                     |
| t |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| m |       |   |       |                     |   |                     |
| R |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| q |       |   |       |                     |   |                     |
| u |       |   |       |                     |   |                     |
| i |       |   |       |                     |   |                     |
| r |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| m |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| n |       |   |       |                     |   |                     |
| t |       |   |       |                     |   |                     |
| s |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
|   |       |   |       |                     |   |                     |
| N |       |   |       |                     |   |                     |
| O |       |   |       |                     |   |                     |
| A |       |   |       |                     |   |                     |
| A |       |   |       |                     |   |                     |
| / |       |   |       |                     |   |                     |
| N |       |   |       |                     |   |                     |
| E |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| D |       |   |       |                     |   |                     |
| I |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| ( |       |   |       |                     |   |                     |
| T |       |   |       |                     |   |                     |
| B |       |   |       |                     |   |                     |
| D |       |   |       |                     |   |                     |
| ) |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| r |       |   |       |                     |   |                     |
| i |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| s |       |   |       |                     |   |                     |
+---+-------+---+-------+---------------------+---+---------------------+
| D |       |   |       |                     |   |                     |
| O |       |   |       |                     |   |                     |
| C |       |   |       |                     |   |                     |
| U |       |   |       |                     |   |                     |
| M |       |   |       |                     |   |                     |
| E |       |   |       |                     |   |                     |
| N |       |   |       |                     |   |                     |
| T |       |   |       |                     |   |                     |
| N |       |   |       |                     |   |                     |
| O |       |   |       |                     |   |                     |
| : |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| C |       |   |       |                     |   |                     |
| L |       |   |       |                     |   |                     |
| A |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 2 |       |   |       |                     |   |                     |
| 0 |       |   |       |                     |   |                     |
| 0 |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| T |       |   |       |                     |   |                     |
| B |       |   |       |                     |   |                     |
| D |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
+---+-------+---+-------+---------------------+---+---------------------+
| * |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| C |       |   |       |                     |   |                     |
| H |       |   |       |                     |   |                     |
| A |       |   |       |                     |   |                     |
| N |       |   |       |                     |   |                     |
| G |       |   |       |                     |   |                     |
| E |       |   |       |                     |   |                     |
| P |       |   |       |                     |   |                     |
| A |       |   |       |                     |   |                     |
| G |       |   |       |                     |   |                     |
| E |       |   |       |                     |   |                     |
| H |       |   |       |                     |   |                     |
| I |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| T |       |   |       |                     |   |                     |
| O |       |   |       |                     |   |                     |
| R |       |   |       |                     |   |                     |
| Y |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
+---+-------+---+-------+---------------------+---+---------------------+
| V | Page  |   | U     |                     |   | Reason for Change   |
| e | Numb  |   | pdate |                     |   |                     |
| r | er(s) |   | In    |                     |   |                     |
| . |       |   | struc |                     |   |                     |
|   |       |   | tions |                     |   |                     |
|   |       |   | (Inse |                     |   |                     |
|   |       |   | rt/De |                     |   |                     |
|   |       |   | lete/ |                     |   |                     |
|   |       |   | Repla |                     |   |                     |
|   |       |   | ce)\* |                     |   |                     |
+---+-------+---+-------+---------------------+---+---------------------+
| 1 | 32-33 |   | Se    |                     |   | Better describe     |
| . |       |   | ction |                     |   | capabilities for    |
| 5 | 6-end |   | 5.3.1 |                     |   | spatial search.     |
|   |       |   | up    |                     |   |                     |
| 2 | 1-24  |   | dated |                     |   | Clarify             |
| . |       |   | with  |                     |   | presentation of     |
| 0 | 1-25  |   | new   |                     |   | requirements        |
|   |       |   | text. |                     |   |                     |
| 2 |       |   |       |                     |   | Response to         |
| . |       |   | Major |                     |   | recommendations of  |
| 1 |       |   | reor  |                     |   | NOAA DMIT           |
|   |       |   | ganiz |                     |   |                     |
| 2 |       |   | ation |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 2 |       |   | Edit  |                     |   |                     |
|   |       |   | orial |                     |   |                     |
|   |       |   | c     |                     |   |                     |
|   |       |   | orrec |                     |   |                     |
|   |       |   | tions |                     |   |                     |
|   |       |   |       |                     |   |                     |
|   |       |   | Many  |                     |   |                     |
|   |       |   | edit  |                     |   |                     |
|   |       |   | orial |                     |   |                     |
|   |       |   | c     |                     |   |                     |
|   |       |   | orrec |                     |   |                     |
|   |       |   | tions |                     |   |                     |
+---+-------+---+-------+---------------------+---+---------------------+
| C |       |   |       |                     |   |                     |
| O |       |   |       |                     |   |                     |
| M |       |   |       |                     |   |                     |
| M |       |   |       |                     |   |                     |
| E |       |   |       |                     |   |                     |
| N |       |   |       |                     |   |                     |
| T |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| : |       |   |       |                     |   |                     |
+---+-------+---+-------+---------------------+---+---------------------+
| N |       |   |       |                     |   |                     |
| O |       |   |       |                     |   |                     |
| T |       |   |       |                     |   |                     |
| E |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| : |       |   |       |                     |   |                     |
+---+-------+---+-------+---------------------+---+---------------------+
| \ |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| E |       |   |       |                     |   |                     |
| X |       |   |       |                     |   |                     |
| A |       |   |       |                     |   |                     |
| M |       |   |       |                     |   |                     |
| P |       |   |       |                     |   |                     |
| L |       |   |       |                     |   |                     |
| E |       |   |       |                     |   |                     |
| S |       |   |       |                     |   |                     |
| : |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| I |       |   |       |                     |   |                     |
| n |       |   |       |                     |   |                     |
| s |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| r |       |   |       |                     |   |                     |
| t |       |   |       |                     |   |                     |
| c |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| n |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| p |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| s |       |   |       |                     |   |                     |
| 6 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 2 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 6 |       |   |       |                     |   |                     |
| t |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| r |       |   |       |                     |   |                     |
| o |       |   |       |                     |   |                     |
| u |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| 6 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 2 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 9 |       |   |       |                     |   |                     |
| f |       |   |       |                     |   |                     |
| o |       |   |       |                     |   |                     |
| l |       |   |       |                     |   |                     |
| l |       |   |       |                     |   |                     |
| o |       |   |       |                     |   |                     |
| w |       |   |       |                     |   |                     |
| i |       |   |       |                     |   |                     |
| n |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| p |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| 6 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 2 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 5 |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
|   |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| R |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| p |       |   |       |                     |   |                     |
| l |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| c |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| p |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| s |       |   |       |                     |   |                     |
| 3 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 1 |       |   |       |                     |   |                     |
| t |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| r |       |   |       |                     |   |                     |
| o |       |   |       |                     |   |                     |
| u |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| 3 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 1 |       |   |       |                     |   |                     |
| 0 |       |   |       |                     |   |                     |
| w |       |   |       |                     |   |                     |
| i |       |   |       |                     |   |                     |
| t |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| c |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| n |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| p |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| s |       |   |       |                     |   |                     |
| 3 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 1 |       |   |       |                     |   |                     |
| t |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| r |       |   |       |                     |   |                     |
| o |       |   |       |                     |   |                     |
| u |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| 3 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 1 |       |   |       |                     |   |                     |
| 0 |       |   |       |                     |   |                     |
| b |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
|   |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
| R |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| p |       |   |       |                     |   |                     |
| l |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| c |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| p |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 5 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 2 |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| w |       |   |       |                     |   |                     |
| i |       |   |       |                     |   |                     |
| t |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| c |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| n |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| p |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 5 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 2 |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| ; |       |   |       |                     |   |                     |
| d |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| l |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| t |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| p |       |   |       |                     |   |                     |
| a |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| e |       |   |       |                     |   |                     |
| s |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 5 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 2 |       |   |       |                     |   |                     |
| 5 |       |   |       |                     |   |                     |
| t |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| r |       |   |       |                     |   |                     |
| o |       |   |       |                     |   |                     |
| u |       |   |       |                     |   |                     |
| g |       |   |       |                     |   |                     |
| h |       |   |       |                     |   |                     |
| 4 |       |   |       |                     |   |                     |
| . |       |   |       |                     |   |                     |
| 5 |       |   |       |                     |   |                     |
| - |       |   |       |                     |   |                     |
| 3 |       |   |       |                     |   |                     |
| 0 |       |   |       |                     |   |                     |
| * |       |   |       |                     |   |                     |
+---+-------+---+-------+---------------------+---+---------------------+

Version Description Record

+------+------------+---+-----------+---+------+--------------+------------+
| DOCU |            |   |           |   |      |              |            |
| MENT |            |   |           |   |      |              |            |
| TI   |            |   |           |   |      |              |            |
| TLE: |            |   |           |   |      |              |            |
|      |            |   |           |   |      |              |            |
| **C  |            |   |           |   |      |              |            |
| LASS |            |   |           |   |      |              |            |
| Arc  |            |   |           |   |      |              |            |
| hive |            |   |           |   |      |              |            |
| and  |            |   |           |   |      |              |            |
| Ac   |            |   |           |   |      |              |            |
| cess |            |   |           |   |      |              |            |
| Sy   |            |   |           |   |      |              |            |
| stem |            |   |           |   |      |              |            |
| Re   |            |   |           |   |      |              |            |
| quir |            |   |           |   |      |              |            |
| emen |            |   |           |   |      |              |            |
| ts** |            |   |           |   |      |              |            |
|      |            |   |           |   |      |              |            |
| NOA  |            |   |           |   |      |              |            |
| A/NE |            |   |           |   |      |              |            |
| SDIS |            |   |           |   |      |              |            |
| (    |            |   |           |   |      |              |            |
| TBD) |            |   |           |   |      |              |            |
| Se   |            |   |           |   |      |              |            |
| ries |            |   |           |   |      |              |            |
+------+------------+---+-----------+---+------+--------------+------------+
| DOCU |            | S |           | D |      |              |            |
| MENT |            | Y |           | O |      |              |            |
| NUM  |            | S |           | C |      |              |            |
| BER: |            | T |           | U |      |              |            |
|      |            | E |           | M |      |              |            |
| Base |            | M |           | E |      |              |            |
| line |            | : |           | N |      |              |            |
| :D   |            |   |           | T |      |              |            |
| raft |            |   |           | B |      |              |            |
| 1.4  |            |   |           | A |      |              |            |
|      |            |   |           | S |      |              |            |
| Curr |            |   |           | E |      |              |            |
| ent: |            |   |           | L |      |              |            |
| 2.0  |            |   |           | I |      |              |            |
|      |            |   |           | N |      |              |            |
|      |            |   |           | E |      |              |            |
|      |            |   |           | I |      |              |            |
|      |            |   |           | S |      |              |            |
|      |            |   |           | S |      |              |            |
|      |            |   |           | U |      |              |            |
|      |            |   |           | E |      |              |            |
|      |            |   |           | D |      |              |            |
|      |            |   |           | A |      |              |            |
|      |            |   |           | T |      |              |            |
|      |            |   |           | E |      |              |            |
|      |            |   |           | : |      |              |            |
|      |            |   |           | 1 |      |              |            |
|      |            |   |           | 0 |      |              |            |
|      |            |   |           | / |      |              |            |
|      |            |   |           | 1 |      |              |            |
|      |            |   |           | 2 |      |              |            |
|      |            |   |           | / |      |              |            |
|      |            |   |           | 0 |      |              |            |
|      |            |   |           | 4 |      |              |            |
+------+------------+---+-----------+---+------+--------------+------------+
| **   |            |   |           |   |      |              |            |
| DOCU |            |   |           |   |      |              |            |
| MENT |            |   |           |   |      |              |            |
| CH   |            |   |           |   |      |              |            |
| ANGE |            |   |           |   |      |              |            |
| H    |            |   |           |   |      |              |            |
| ISTO |            |   |           |   |      |              |            |
| RY** |            |   |           |   |      |              |            |
+------+------------+---+-----------+---+------+--------------+------------+
| DCN  | Revis      |   | Date      |   | DCN  | Rev          | Date       |
| No.  | ion/Update |   |           |   | No.  | ision/Update |            |
|      | Nos.       |   |           |   |      | Nos.         |            |
+------+------------+---+-----------+---+------+--------------+------------+
|      | Draft,     |   | 9/16/04   |   |      |              |            |
|      | Version    |   |           |   |      |              |            |
|      | 1.0        |   | 9/25/04   |   |      |              |            |
|      |            |   |           |   |      |              |            |
|      | Draft,     |   | 10/5/04   |   |      |              |            |
|      | Version    |   |           |   |      |              |            |
|      | 1.1        |   | 10/6/04   |   |      |              |            |
|      |            |   |           |   |      |              |            |
|      | Draft,     |   | 10/12/04  |   |      |              |            |
|      | Version    |   |           |   |      |              |            |
|      | 1.2        |   | 2/23/05   |   |      |              |            |
|      |            |   |           |   |      |              |            |
|      | Draft,     |   | 4/22/05   |   |      |              |            |
|      | Version    |   |           |   |      |              |            |
|      | 1.3        |   | 4/26/05   |   |      |              |            |
|      |            |   |           |   |      |              |            |
|      | Draft,     |   | 5/16/05   |   |      |              |            |
|      | Version    |   |           |   |      |              |            |
|      | 1.4        |   |           |   |      |              |            |
|      |            |   |           |   |      |              |            |
|      | Draft,     |   |           |   |      |              |            |
|      | Version    |   |           |   |      |              |            |
|      | 1.5        |   |           |   |      |              |            |
|      |            |   |           |   |      |              |            |
|      | Version    |   |           |   |      |              |            |
|      | 2.0        |   |           |   |      |              |            |
|      |            |   |           |   |      |              |            |
|      | Version    |   |           |   |      |              |            |
|      | 2.1        |   |           |   |      |              |            |
|      |            |   |           |   |      |              |            |
|      | Version    |   |           |   |      |              |            |
|      | 2.2        |   |           |   |      |              |            |
+------+------------+---+-----------+---+------+--------------+------------+
| NO   |            |   |           |   |      |              |            |
| TES: |            |   |           |   |      |              |            |
+------+------------+---+-----------+---+------+--------------+------------+

**Table of Contents**

1.  **Introduction** 1

    1.  Background 1

    2.  Purpose and Scope of this Document 1

    3.  Document Organization 1

    4.  The Vision for CLASS 2

    5.  Major Features 3

    6.  Challenges 3

    7.  Assumptions and Dependencies 4

    8.  Archive Requirements Working Group 4

    9.  Frequently-Used Acronyms and Abbreviations 6

    10. Acknowledgements 7

2.  **Data Stewardship** 7

    1.  Scientific Data Stewardship 7

    2.  Open Archival Information System 8

    3.  Reprocessing 8

    4.  Stewardship Requirements 10

3.  **User Requirements** 12

    1.  Designated User Community 12

    2.  CLASS User Working Group 12

    3.  Data Discovery 13

    4.  Data and Product Delivery 20

    5.  Subscriptions 23

4.  **Federal Requirements for Archive Records Management** 24

5.  **Producer Requirements** 25

    1.  Submission Agreements 25

6.  **Management Requirements** 26

    1.  Configuration Management 26

    2.  Customer Feedback 26

    3.  Usage Metrics 26

7.  **Priorities for Action** 27

**Appendices**

Appendix 1 Procedures for Negotiation of a Submission Agreement 29

Appendix 2 Related Data Management Activities 36

Appendix 3 Relevant Guidelines and Standards 43

Appendix 4 Applicable Documents and References 55

Appendix 5 OAIS Reference Model 56

Appendix 6 CLASS Dataset Responsibilities 59

# Introduction

## Background

The National Environmental Satellite, Data, and Information Service
(NESDIS) is responsible for the collection, archive, and dissemination
of environmental data collected by a variety of in situ and remote
sensing observing systems operated by the National Oceanic and
Atmosphere Administration (NOAA) and by a number of its national and
international partners, such as the National Aeronautics and Space
Administration (NASA) and the European Organization for the Exploitation
of Meteorological Satellites (EUMETSAT). To prepare for large increases
in the volume and diversity of these data holdings, NESDIS initiated the
planning and development for a Comprehensive Large Array-data
Stewardship System (CLASS) that provides archive and access services for
these data.

The Satellite Active Archive served as the foundation for the archive,
access, and distribution functionality of CLASS. Enhancements to this
baseline system are required to better support data providers and
customers with improved performance and availability as well as
additional functionality. These enhancements must be responsive to the
requirements of the CLASS data providers and customers and must also
meet requirements implicit in federal mandates and resulting from
conformance to applicable national and international standards.

## Purpose and Scope of this Document 

The Archive Requirements Working Group (ARWG) has been established to
ensure that science and other user requirements are clearly defined with
respect to NOAA's archive, access, and reprocessing stewardship
activities and to serve as a clearinghouse for requirement planning.
This document describes initial requirements for further CLASS
development recommended by the ARWG. These requirements have either been
explicitly defined by CLASS data providers or users or are implicitly
required for effective and responsible data stewardship.

The requirements included within this document are directly related to
data archive, access, or reprocessing. Requirements specifically
pertaining to telemetry, data ingest and the standard
[procedures]{.underline} for quality control of data managed by CLASS
are not considered. (But computational requirements for quality control
performed as aspect of reprocessing are considered.)

Many specific access and archive requirements for CLASS are already
defined in the CLASS Archive and Access Requirements document (Computer
Sciences Corporation, July 20, 2001) or CLASS Allocated Requirements
(CLASS-1017-CLS-REQ-AADS). Since most of these specific requirements
have already been accommodated in the operational system, they are not
included nor prioritized within this document.

## Document Organization

This requirements document is organized as follows.

Section 1 -- An introduction to CLASS, the purpose and scope of this
document, challenges and other background material

Section 2 - General requirements for data stewardship and an
introduction to the Open Archive Information System model

Section 3 - User requirements for data discovery, access, delivery and
use

Section 4 Federal Requirements for archive records management

Section 5 - Producer requirements for the CLASS archival and access
system

Section 6 - Management Requirements (e.g. configuration control, usage
metrics)

Section 7 - Priorities for Action

Appendices -- 1 Procedures for Negotiation of a Submission Agreement

> 2 Related Data Management Activities
>
> 3 Relevant Guidelines and Standards
>
> 4 Applicable Documents and References
>
> 5 OAIS Reference Model
>
> 6 CLASS Dataset Responsibilities

## The Vision for CLASS

CLASS supports the NESDIS mission to acquire, archive, and disseminate
environmental data. NESDIS has been acquiring these data for more than
30 years, from a variety of in situ and remote sensing observing systems
operated by NOAA and from a number of its partners. NESDIS foresees
significant growth in both the data volume and the user population for
these data, and has therefore initiated this effort to evolve current
technologies to meet future needs.

NOAA\'s National Data Centers and their world-wide clientele of
customers look to CLASS as the primary NOAA information technology
infrastructure project in which all current and future large array
environmental data sets will reside. CLASS provides permanent, secure
storage and safe, efficient access between the Data Centers and the
customers. The initial objective for CLASS is to provide storage,
archival and access for large-array data sets, specifically from the
following campaigns:

-   NOAA and Department of Defense Polar-orbiting Operational
    Environmental Satellites (POES) and Defense Meteorological Satellite
    Program (DMSP)

-   NOAA Geostationary-orbiting Operational Environmental Satellites
    (GOES)

-   National Aeronautics and Space Administration (NASA) Earth Observing
    System (EOS) Moderate-resolution Imaging Spectrometer (MODIS)

-   National Polar-orbiting Operational Environmental Satellite System
    (NPOESS)

-   The NPOESS Preparatory Program (NPP)

-   EUMETSAT Meteorological Operational Satellite (Metop) Program

-   NOAA NEXt generation weather RADAR (NEXRAD) Program

-   NCEP NWP Model Datasets

The requirements defined within this document pertain to CLASS managing
data and products from only these eight campaigns. Even with this
limitation, these requirements set out a mission that will take many
years to build successfully. Should the mission of CLASS be expanded to
include additional campaigns and data types, many additional
requirements would need to be considered and addressed.

CLASS is currently supported by the Climate Goal. Although it is
recognized that CLASS serves the needs of all NOAA customers, the
requirements of the Climate Goal take priority. In particular, CLASS
must be able to ingest, archive and provide access to long-term
satellite climate data records produced from these large-array data
sources, both existing and those to be defined in the future (see
Appendix 7).

CLASS will build upon systems already in place to contribute to an
architecture for an integrated, national environmental data access and
archive system to support a comprehensive data management strategy.
Number range CHAPTERThe goals of CLASS are as follows:

1.  Give any potential customer access to all NOAA (and some selected
    non-NOAA) large-array data through a single portal.

2.  Eliminate the need to continue creating "stovepipe" systems for each
    new type of data, while, as much as possible, using already refined
    portions/modules of existing legacy systems.

3.  Define and implement a cost-effective architecture that can
    primarily handle large array-data sets, but should be adaptable and
    expandable to handle other types of data sets as well.

4.  Support the processing and reprocessing of any or all datasets
    managed by CLASS.

The development of CLASS is expected to be a long-term, evolutionary
process, as current and new campaigns are incorporated into CLASS.
Therefore, this document is expected to evolve as additional datasets
are incorporated into CLASS and as technology changes.

## Major Features

An important goal of CLASS is to provide a major portal for access to
NOAA environmental data, some of which is stored in CLASS itself, and
some available from other archives. The most significant processes
required to meet this goal that are within the scope of CLASS are:

-   Ingest of environmental data from CLASS data providers

-   Extraction, storage, and provision of metadata describing the data
    stored in CLASS

-   Archiving data

-   Browse and search capability to assist users in finding data

-   Distribution of CLASS data in response to user requests

-   Identification and location of environmental data that is not stored
    within CLASS, and links to the responsible system

-   Charging for data, as appropriate ^\#^

```{=html}
<!-- -->
```
-   Operational support processes: disaster recovery, help desk/CLASS
    support

-   Maintaining a user statistics data base and providing standard and
    ad hoc statistical reports of CLASS users

\# - While the capability of charging for delivery of data via portable
media is a requirement for CLASS, the development of an e-Commerce
system to support financial transactions is out of the scope of CLASS.
CLASS will interface with the NESDIS e-commerce System for financial
transactions.

## Challenges

The Satellite Active Archive (SAA) has served as the foundation for the
archive, access, and distribution functionality for CLASS. The SAA was
established as a demonstration prototype for electronic distribution of
POES data in 1994 and became operational in July 1995. During that first
month, 379 Advanced Very High Resolution Radiometer (AVHRR) Level 1b
data sets were distributed to 27 customers via the emerging Internet. In
the nine years since, average monthly volume has increased to nearly
220,000 files and the SAA (now CLASS) customer base stands at more than
23,000 active, registered customers. In FY 2003, CLASS electronically
distributed more than 26 terabytes of polar satellite data, Synthetic
Aperture Radar (SAR) data, Coast Watch data, and derived data products
to its customers. CLASS has more than 5.7 million data files on-line or
near-line, including 88% of all NOAA AVHRR and TIROS Operational
Vertical Sounder (TOVS) data and DMSP data and 100% of all NOAA Coast
Watch data.

Enhancements to the SAA baseline system are required in order to support
existing and new data providers and customers, including improved
performance and availability as well as additional functionality. The
CLASS development team must establish an operational environment that
permits the infusion of new, improved access and distribution
technologies and the introduction of data from additional campaigns
with: 1) no negative impact on current customer satisfaction; 2) minimal
impact on future operational funding; and 3) continuing improvement in
the amount and quality of data and derived data products available
through CLASS.

GEOSS

The Group on Earth Observations has adopted the Global Earth Observation
System of Systems (GEOSS) 10-Year Implementation Plan. The Plan builds
on and adds value to existing Earth observation systems by coordinating
their efforts, addressing critical gaps, supporting their
interoperability, sharing information, reaching a common understanding
of user requirements and improving delivery of information to users.

As noted in the plan, the success of GEOSS will depend on
interoperability between the participating data and information
providers. GEOSS interoperability will be based on non-proprietary
standards, with preference to formal international standards.
Observations and products contributed and shared within GEOSS should be
recorded and stored in clearly defined formats, with metadata and
quality indications to enable search, retrieval, and archiving as easily
accessible data sets. The GEOSS Plan stresses the importance of using
existing international standards organizations and institutes as a focal
point for the GEOSS interoperability objectives as they relate to and
use standards.

NOAA was one of the founders and driving forces behind the development
of the GEOSS Plan and NOAA is committed to being a leader in its
implementation. Consequently, NOAA data management systems must ensure
they are compatible and interoperable with systems developed or operated
by other GEOSS partners. As a major component of NOAA's information
technology infrastructure, CLASS must place interoperability with other
environmental information systems as one of its top priorities.

Requirement 1.1 CLASS must ensure interoperability between the data and
products that it manages and other data sources and data types. To
achieve this goal, it should conform to all relevant federal and
international standards that apply to the collection, management,
discovery and dissemination of environmental data. It should also strive
to conform to emerging community and industry standards that contribute
to interoperability. Details on these standards are provided in specific
requirements described later in this document.

## Assumptions and Dependencies

As noted in the CLASS Five-Year Plan (CLASS-1013-CLS-PLN-5YEAR), future
plans for CLASS must allow for increasing demands from customers. CLASS
must provide faster access to data, easier browsing and ordering of
data, and the ability to access products derived from the data sets of
interest. CLASS personnel will work with the ARWG to ensure these
enhancements are responsive to customer requirements

The reach and capacity of the Internet have increased tremendously over
the past decade. The capabilities of Internet applications, such as
browsers, have shown steady growth and the Internet is increasingly
being used for business-critical applications. These trends are expected
to continue.

The business, scientific, education and government communities have all
recognized that the rise of the Internet presents numerous opportunities
for increased services, improved efficiency and lower costs. However, to
realize the full potential of the Internet requires development and
implementation of standards: standard protocols, standard interfaces,
and standard terminology. Standards to facilitate information exchange
are being continuously developed and implemented by both the public and
private sectors. To provide the best service to its customers, CLASS
must keep abreast of advances in relevant standards and must be an
active participant and partner with other groups in NOAA, the scientific
community, and industry in further development of these standards..

## Archive Requirements Working Group

NOAA is focused on developing an integrated global environmental
observation and data management system as part of its commitment to
serve the Nation. As one of the first steps toward this goal, NOAA
established the NOAA Observing System Council (NOSC) in May 2003. NOSC
is the principal advisory body to the NOAA Executive Council for earth
observation and data management activities and the principal
coordinating body for NOAA to the White House Committee on Environment
and Natural Resources Subcommittee on Earth Observations in developing a
comprehensive, coordinated and sustained Earth observation system. The
NOAA Data Management Committee (DMC) (formerly the NESDIS Data Archive
Board) was established under NOSC to provide clear guidance for managing
NOAA data stewardship and to provide the NOSC with the information it
needs to integrate data stewardship with the NOAA Observing Systems
Architecture and into the NOAA data management enterprise.

The ARWG was established under the Data Archive Board (a NESDIS Line
Office structure now defunct) to ensure that science requirements and
other user applications are clearly defined with respect to NOAA's
archive, access, and reprocessing stewardship activities. The ARWG now
reports to the DMC.

The ARWG serves as a clearinghouse for requirement planning in support
of the science objectives related to archive, access, and reprocessing
with an initial focus on large-array datasets (satellite, model and
radar) and other holdings of the NOAA National Data Centers.
Requirements are defined, validated and then presented for review and
approval. The ARWG provides advice and assistance to facilitate the
definition and implementation of a NOAA stewardship architecture that
balances the needs of users for a dynamic and flexible system and the
needs of developers for structure and continuity.

Terms of reference

-   To ensure that CLASS and other stewardship activities under its
    purview are responsive to the goals of the NOAA strategic plan

-   To provide for effective use of stewardship activities by ensuring
    that mechanisms are in place to engage the user community in
    decisions concerning what data and products are to be included in or
    excluded from the NOAA National Data Center archives

-   To periodically review the contents of the NOAA National Data Center
    archives for disposition in conjunction with NARA guidelines and
    data center record retention and disposition schedules.

-   To provide requirements and prioritization for preserving and
    maintaining the basic storage of and access to critical large
    array-data, other NOAA National Data Center critical data sets and
    derived products and their documentation, including verifying their
    quality and compliance with federal standards

-   To establish requirements for the information technology aspects for
    implementation of scientific data stewardship

-   To provide a long-term vision to incorporate other NOAA data
    holdings into a system of archive, access, and scientific data
    stewardship

The relationship between these groups is illustrated in Figure 1.1
below.

![](media/image1.wmf){width="5.9944444444444445in"
height="3.9479166666666665in"}

Figure 1.1 CLASS Project responsibilities (proposed to Data Management
Committee)

## Frequently Used Acronyms and Abbreviations

  ------------ ----------------------------------------------------------
  ANSI         American National Standards Institute

  API          Application Programmer Interface

  ARWG         Archive Requirements Working Group

  AVHRR        Advanced Very High Resolution Radiometer

  CCSDS        Consultative Committee for Space Data Systems

  CDR          Climate Data Records

  CLASS        Comprehensive Large Array-data Stewardship System

  CSDGM        Content Standard for Digital Geospatial Metadata

  DMAC         Data Management and Communications (component of IOOS)

  DMSP         Defense Meteorological Satellite Program

  EOS          Earth Observing System

  EOSDIS       Earth Observing System Data and Information System

  FGDC         Federal Geographic Data Committee

  FTP          File Transfer Protocol

  GCMD         Global Change Master Directory

  GeoTIFF      Geographic TIFF

  GIF          Graphics Interchange Format

  GIS          Geographic Information System

  GOES         Geostationary-orbiting Operational Environmental
               Satellites

  HDF          Hierarchical Data Format

  HTTP         Hypertext Transport Protocol

  IOOS         Integrated Ocean Observing System

  ISO          International Organization for Standardization

  JPEG         Joint Photographic Experts Group

  MPEG         Moving Picture Experts Group

  NASA         National Aeronautics and Space Administration

  NCDC         National Climatic Data Center

  NCDDC        National Coastal Data Development Center

  NESDIS       National Environmental Satellite, Data, and Information
               Service

  netCDF       Network Common Data Form

  NEXRAD       NEXt generation weather Radar

  NGDC         National Geophysical Data Center

  NOAA         National Oceanic and Atmosphere Administration

  NODC         National Oceanographic Data Center

  NOSC         NOAA Observing System Council

  NSDI         National Spatial Data Infrastructure

  OAIS         Open Archival Information System

  OGC          Open GIS Consortium

  OPeNDAP      Open source Project for a Network Data Access Protocol

  PDI          Preservation Description Information

  POES         Polar-orbiting Operational Environmental Satellites

  SAA          Satellite Active Archive

  SDS          Scientific data stewardship

  SOAP         Simple Object Access Protocol

  TOVS         TIROS Operational Vertical Sounder

  WSDL         Web Services Description Language

  XML          Extensible Markup Language
  ------------ ----------------------------------------------------------

## Acknowledgements

This document builds upon the work of many data management experts who
have prepared the references listed in Appendix 4. Particular thanks are
extended to the authors of *Data Management and Communications Plan for
Research and Operational Integrated Ocean Observing Systems*, as many of
the requirements listed in this document have been adapted from that
plan.

# Climate Data Stewardship

## Scientific Data Stewardship

Scientific data stewardship is the modern paradigm in data management,
consisting of an integrated suite of functions to preserve and exploit
the full scientific value of environmental data. These functions
include:

-   Careful monitoring of observing system performance for long-term
    applications

-   Generation of authoritative long-term records from multiple
    observing platforms

-   Assessment of the state of the atmospheric, oceanic, land,
    cryospheric and space environments

-   Integration of data from multiple observing platforms and other
    sources

-   Proper archival of and timely access to data and metadata.

Successful implementation of scientific data stewardship will ensure
NOAA's environmental data are of maximum use to the Nation, now and in
the future.

Since real-time operational products from NOAA's space and ground-based
environmental observing systems are intended to satisfy its short term
environmental forecasting goals, they have not always been sufficiently
accurate or calibrated to serve as authoritative long-term records.
These long-term records are needed to address important environmental
monitoring and prediction issues such as atmospheric and oceanic climate
change, terrestrial change detection, space and solar variability, and
ecosystem and coastal management.

Within the generic construct of data stewardship, the CLASS and
Scientific Data Stewardship (SDS) programs have specifically been set up
to apply the principles of data stewardship to NOAA's large satellite
data holdings and to provide a focal point for the application of data
stewardship principles more generically. The CLASS and SDS programs have
been designed to dovetail into one another: CLASS provides for the
information technology aspects and SDS provides the science input. The
following sections describe the needs for SDS within the CLASS program.

SDS, which fuses satellite and ground-based measurements, provides an
end-to-end system that addresses these limitations. SDS can correct many
data problems identified by the scientific community and permit more
significant applications to economic and social issues by focusing on
NOAA's environmental stewardship mission. To fully exploit the
scientific value of NOAA data by current and future users, four
functions, must be achieved.

a.  The observing system performance with respect to long-term
    applications must be monitored in real-time. Such monitoring
    requires the establishment of tracking tools necessary for the
    detection of subtle spatial and temporal biases in the observing
    system as well as in the observation record. These biases can then
    be minimized or eliminated through coordination among network
    operators.

b.  Authoritative, long-term records must be generated. This function
    will preserve and enhance the value of the irreplaceable historical
    data by conducting rigorous data analysis and research to validate
    and improve these records, and by reprocessing and enabling others
    to reprocess the entire data stream from the fundamental
    measurements when scientific advances warrant it. SDS will use
    physics-based techniques to fuse together data from disparate
    observing systems, such as direct measurements from ground-based
    networks and remotely sensed measurements from satellite
    instruments. The authoritative nature and vitality of these products
    will be maintained through periodic reviews. These authoritative
    long-term records encompass the concept of a Climate Data Record
    described in section 2.3.

c.  These authoritative records can be used to assess the current state
    of the environment and to put it in historical perspective.
    Long-term trends on local, regional or global scales can be
    determined and estimates for the future developed. In addition the
    authoritative records can be used to detect changes in environmental
    conditions between different time periods and different
    environmental regimes.

d.  To ensure complete archival and access capabilities, metadata,
    direct observations, and fundamental records from satellite and in
    situ platforms must be comprehensive, complete and preserved in
    perpetuity. Open efficient access to the metadata, products, and
    data streams must be ensured, and data made available in
    well-documented and useful formats.

## Open Archival Information System

The International Organization for Standardization (ISO) has encouraged
the development of standards in support of the long-term preservation of
digital information obtained from observations of the terrestrial and
space environments. ISO has requested the Consultative Committee for
Space Data Systems (CCSDS) Panel 2 to coordinate the development of
those standards.

The initial effort has been the development of a Reference Model for an
Open Archival Information System (OAIS, CCSDS 650.0-B-1). The OAIS
Reference Model has been extensively reviewed and, pending some
editorial updates, has been approved as an ISO Standard and as a CCSDS
Recommendation. Additional information on the OAIS Reference Model is
provided in Appendix 6 and the complete copy of the latest version is
available at <http://www.ccsds.org/documents/650x0b1.pdf>.

## Reprocessing

Climate data records (CDR) will be generated from low-level data sets
(levels 0 and 1) that are archived within CLASS. CDRs require that
systematic errors in the data or any artifacts in the CDR time series,
due to for example, changes in the algorithm or processing system, be
minimized. This requirement arises because the signals of interannual
variability and decadal trends are very small and artificial jumps in
the time series can obscure the natural signal that we are trying to
detect. Thus, a periodic reprocessing of the entire time series of data
from various satellite instruments will be required and CLASS must
accommodate requests for data needed for this reprocessing. Such
requests, however, will originate through the Scientific Data
Stewardship (SDS) project and the implementation and administrative
plans for that project will not be available until later in FY05.

The best way to achieve an initial operating capacity for CLASS to
support reprocessing prior to implementation of the SDS program is by
formulating a set of pilot projects or pathfinder projects. Each pilot
would aim to exercise and evaluate a different aspect of reprocessing
procedures and requirements.

The original NOAA/NASA Pathfinder project of the early 1990s was the
initial impetus for the formation of the satellite active archive, the
predecessor to CLASS. NASA used the Pathfinder as a way to ramp up to
the processing and reprocessing that would be needed under the Earth
Observing System Data and Information System (EOSDIS). NOAA should use a
similar approach in preparation for the reprocessing that will be needed
in the NPP/NPOESS era. The largest volume satellite data sets currently
in CLASS are the GOES data, and hence they would be good candidates for
such a trial effort. There are also user communities who have
requirements for reprocessing of GOES data, including products for
precipitation, sea surface temperature, cloud drift winds and more. The
SDS program will provide the organizational structure for holding a
workshop to engage these user communities to obtain requirements for
CLASS to optimize access to large volumes of data (instead of having
multiple individual requests for massive amounts of data). We believe
that such a process will serve as an excellent pilot project and
learning experience.

A new version of the AVHRR SST Pathfinder reprocessing effort would also
be a good pilot. An AVHRR SST pathfinder reprocessing effort is
currently implemented through a grant to the University of Miami.
However, this capability should be transitioned to operations and
migrated to NOAA to ensure a sustained effort. This would be a valuable
test of the CLASS system and its ability to support reprocessing of
AVHRR data.

Additional pilots should be formulated and evaluated by the SDS program.

In the longer term, we anticipate that CLASS reprocessing requirements
will evolve to be quite similar to those for the NASA EOS, so the
current EOSDIS system can be examined as a placeholder for anticipated
SDS requirements for NPP/NPOESS data reprocessing in the future. The
largest data rate instruments on the EOS platforms are the MODIS
instruments and the VIIRS instruments on the NPP and NPOESS platforms
will have a similar data rate for level 0 and level 1 data sets. Science
teams set CDR requirements for MODIS reprocessing and we expect a
similar organization of science teams will emerge from the SDS program.
A central processing facility then processes level 0 and 1 data into
level 2 products. These duties of a central processing facility are
defined as requirements for SDS not CLASS, although it is clear that a
close coordination with CLASS will be required. CLASS will be
responsible for serving up data from the archive and archiving the
processed level 2 CDR. CLASS would also be responsible for procurement
of the IT hardware resources for CDR processing. To illustrate the
high-level data flow for operational processing and reprocessing, a data
flow diagram of MODIS processing is provided below.

Data flow rates and processing volumes are nominally expressed are rates
per day at 1X operational ingest or processing. These rates can be
increased as processing power increases with Moore's law and, in order
to allow for a complete reprocessing of a massive data set, the
reprocessing rate must continue to increase at a rate equal to or
greater than the cumulative data volume. This rate, however, only needs
to handle the lower of level 0 or level 1 data volumes, since processing
speed is much greater than extraction speed from the archive. (Thus, it
is usually faster to read and process level 0 data if it is of lower
volume than level 1 data).

![](media/image2.png){width="6.0in" height="3.9520833333333334in"}

Figure 2.1 MODIS Processing Systems

Another way to accommodate the requirements of the climate community for
reprocessing, as well as to involve a large user community in examining
the NPP and NPOESS data, is to provide easy access to data sets that
have been sub-sampled or otherwise compressed to reduce total data
volume. The SDS program will attempt to identify what levels of data
volume reduction would still serve the requirements of a large fraction
of the user community while reducing the volume by at least one order of
magnitude. Experience has shown that many users will find it easier for
them to initially work with a much reduced volume of data and this will
allow for the gradual increase in the capability to perform large-scale
reprocessing as the SDS program spins up.

Reprocessing activities can place significant demands upon an archive
system. CLASS must be able to supply the huge volume of data needed to
meet reprocessing requirements at a rate that is sufficient to support a
given reprocessing ratio without delaying operational processing. To
provide this capability CLASS must meet the following requirements.

## Stewardship Requirements

Requirement 2.1 The organization and operation of CLASS should follow
the OAIS Reference Model.

Requirement 2.2 CLASS must have sufficient communication and
computational capacity to concurrently ingest operational data, serve
customers and support reprocessing,

Requirement 2.3 CLASS must archive and provide easy access to all
information pertaining to processing of CLASS data, including
documentation of processing algorithms and procedures,

Requirement 2.4 CLASS must provide easy access to validation datasets
(in situ, aircraft, etc) and all ancillary data used in operational
processing.

Requirement 2.5 Access rates for reprocessing of level 0 or level 1 data
(whichever is smaller) must scale at a rate proportional to the total
volume of the data set such that the time to recover the total volume
from CLASS does not exceed 18 months.

Requirement 2.6 IT hardware infrastructure required for SDS for
reprocessing will vary with thematic area (e.g., ocean, atmospheres,
land) as demonstrated by EOSDIS. This processing rate is defined by the
ratio of data-days/production day (i.e., 1x rate) and this ratio must
increase with total volume to be reprocessed so that the entire data
volume of operational polar satellites can be reprocessed within 18
months or less. Current EOSDIS processing rates to achieve this are 9x
(7x reprocessing, 2x forward near real-time) for collection 5 EOSDIS
ongoing in 2004 and 2005. [NOAA may partner with other agencies (NASA,
DOE and others) to achieve the required processing power.]{.underline}

Requirement 2.7 Ingest of CDRs (level 2 and 3 data) should proceed at a
rate approximately equal to that of EOSDIS collection 5 processing. This
rate is approximately equal to the daily rate of production of level 0
and level 1 data.

Requirement 2.8 CLASS must be able to deliver and ingest the volume of
input and output data required for reprocessing. Order of magnitude
estimates are provided in the table below. While recognizing that there
are many sources of data, we have used the VIIRS and GOES numbers
because they are the major drivers for volume requirements.

  ---------------------- ---------- ---------- ---------- ---------- ----------
  **Instrument/level**   **FY       **FY       **FY       **FY       **FY
                         2007**     2008**     2009**     2010**     2011**

  VIIRS level 0          600^1^     900        1200       1500       1800

  VIIRS level 1          2800^2^    4200       5600       7000       8400

  GOES GVAR              480^3^     600        720        840        960

  Cumulative Total       3380^4^    5700       7520       9340       11160
  ---------------------- ---------- ---------- ---------- ---------- ----------

> Processing rates (GB/day) required to reprocess the total archive in
> 12 months
>
> 1\. 150 GB/day per satellite x 4 (2 satellites x 2 processing (1
> operational, 1 reprocessing))
>
> 2\. 700 GB/day x 4
>
> 3\. 30 GB/day x 16 (2 satellites x 2 processing + 12 years existing
> archive since 1996)
>
> 4\. Input volume for processing and reprocessing is equal to the sum
> of VIIRS level 0 and GOES GVAR for each year. Output volume of CDRs
> (i.e., level 2 and 3 data) is equal to the cumulative total each year
> (3380 GB/day in FY07)

Requirement 2.9 CLASS should provide the IT infrastructure for
reprocessing. This requires computational resources (expressed in
Floating Point Operations per second) as well as input/output
capabilities. An estimate of the computational and storage requirements
for reprocessing is provided below.

+-----------------------+-----------------+----------+-----------------+
| **Requirement**       | **Change in     | **       | **Note**        |
|                       | computational   | Expected |                 |
| **Change description  | power           | change   |                 |
| (e.g., increase       | required**      | date**   |                 |
| domain)**             |                 |          |                 |
+-----------------------+-----------------+----------+-----------------+
| Initial operating     | CPU = 25 GFlops | CY06     | This disk farm  |
| capacity for          | & 4 TB RAM, 50  |          | is for CDR      |
| processing and        | TB disk farm,   |          | science and     |
| reprocessing of       | fiber channel   |          | instrument team |
| Climate Data Records  | connection to   |          | use, not CLASS  |
| from current and past | CLASS at NCDC   |          | archive         |
| NOAA operational      |                 |          |                 |
| satellite data        |                 |          |                 |
+-----------------------+-----------------+----------+-----------------+
| Annual CPU and RAM    | CPU = 10 GFlops | CY07     | Annual refresh  |
| refresh to keep pace  | & 1 TB RAM to   | CY08     | provides        |
| with increasing       | Linux cluster   |          | optimal         |
| cumulative data       |                 | CY09     | c               |
| volume                |                 |          | ost/performance |
|                       |                 | CY10     | benefit vs.     |
|                       |                 |          | Moore's law     |
|                       |                 | CY11     |                 |
+-----------------------+-----------------+----------+-----------------+
| Refresh of disk farm  | 100 TB disk     |          |                 |
| every 3 years         | farm            |          |                 |
+-----------------------+-----------------+----------+-----------------+

# User Requirements

In planning future enhancements, CLASS must take customer needs and
expectations into account. Customers expect computer-based systems to
continuously improve so CLASS must keep up with and take advantage of
emerging standards, improvements in information technology, including
faster hardware, expanded Internet bandwidth, and improved software.. It
must evolve to provide faster access to data and easier, interoperable
ways of finding, browsing, ordering, receiving, and using data and
derived products.

## Designated User Community

The community of potential users of CLASS runs the gamut from those with
limited computer knowledge to elementary school teachers (and students)
to internationally recognized experts in remote sensing. Serving such a
heterogeneous community with a single system would present an impossible
challenge. Therefore, CLASS will focus on serving the scientific and
technical community. CLASS customers will be expected to be
scientifically and technically literate and familiar with tools for
analyzing scientific and gridded data. However, they should not need to
have extensive pre-existing experience with satellite instruments or
CLASS datasets.

To serve other users, CLASS should help to enable a value-added "market"
(not necessarily commercial), in which others may produce information
products tailored to particular user groups.

## CLASS User Working Group

CLASS should define a formal mechanism for user outreach to ensure
ongoing communication between CLASS developers and users. It is
recommended that CLASS establish a working group to represent the
interests of CLASS users relating to data archival, processing, and
distribution.

Requirement 3.1 CLASS should establish a CLASS User Working Group to
represent the interests of CLASS users relating to data archival,
processing, and distribution.

The CLASS User Working Group would serve three functions. It would make
recommendations concerning:

a.  Further development of the CLASS interface, both interactive and via
    automated machine to machine procedures

b.  Data processing and the broader functions of the data system, e.g.
    data selection, access and distribution

c.  Prioritization of enhancements to CLASS, with respect to meeting
    science data and information needs

d.  What data and products are to be included in or excluded from the
    NOAA National Data Center archives in general and CLASS in
    particular.

e.  Cultivating a value-added provider community that, through
    generation of tailored products, can make CLASS data useable by
    specialized groups (e.g. educators, health or safety professionals,
    etc.)

Membership should be comprised of the Archive Requirements Working Group
plus additional science data specialists from the CLASS user community.
There should be at least two representatives from each of the government
research, university and private industry communities, as well as
value-added system developers

The working group could interact through a moderated Internet forum,
although an occasional workshop would be advantageous. The CLASS Project
Office should support working group correspondence and report generation
and workshop costs.

## Data discovery 

### Metadata

Understanding complex environmental problems demands data from many
observing systems. In order to be useful to a variety of users, these
data need to be extensively documented. Long-term preservation imposes
the additional requirement that the documentation must be sufficiently
detailed to allow the data to be successfully used even after all
personnel with first hand experience have left the observing program.

Dataset search capabilities also depend upon sufficient and accurate
descriptive Information for datasets. This descriptive information, or
metadata, must be available in accordance with Federal, International,
and industry standards (see Appendix 3). The metadata must be complete
enough for customers to determine if a dataset includes the parameters,
time period and geographic location they are looking for, at the
resolution, frequency, quality and accuracy they require.

As further explained in Appendix 3, Federal regulations and NOAA policy
require that geospatial data be described in accordance with Federal
Geographic Data Committee (FGDC) Content Standard for Digital Geospatial
Metadata (CSDGM). The next version of the FGDC standard (version 2) will
be a form of the International Content Standard for Geographic Metadata
(ISO 19115) so compliance with ISO 19115 should also be a priority.

Both the FGDC-CSDGM and ISO 19115 metadata standards are comprehensive
in their approach. They are, therefore, quite lengthy and complex. The
document defining each one is nearly 100 pages long, not including
accompanying sample implementations. Initially daunted by the complexity
of these standards and their data models, system developers have often
been inclined to select only a portion of the standard that they
determine to be pertinent to their own data system. In many cases, this
is perfectly acceptable, since it can be shown that their needs can be
met with only a limited subset of the entities defined in the standard.
Unfortunately, CLASS cannot adhere to this limited perspective.

CLASS must be able to manage data of long-term historical interest for
an indefinite period of time. Thus, it must be capable of managing the
comprehensive information needed to describe a dataset far into the
future, when there is no longer anyone available with personal
first-hand experience with the data or its processing. With this in
mind, any field within the ISO or FGDC metadata standards relevant to
remotely sensed or gridded data will probably be required for at least
one dataset.

**Metadata Model**

Requirement 3.2 CLASS must formally design a metadata model, management
system and interface for metadata.

Requirement 3.2.1 The metadata model used by CLASS must be able to
manage data in any or all of the fields defined in the FGDC CSDGM or the
ISO International Standard for Geographic Information -- Metadata (ISO
19115), including extensions for imagery and gridded data. FGDC CSDGM is
available at <http://www.fgdc.gov/metadata/> meta_stand.html and ISO
19115 can be purchased at
<http://www.iso.ch/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=26020&ICS1=35>

Requirement 3.2.2 The metadata model used by CLASS must include all
metadata provided to CLASS with NESDIS and NOAA datasets and products.
These metadata must be available to the users as part of the data
discovery system.

Requirement 3.2.3 CLASS should include the extensions of ISO 19115
registered by The World Meteorological Organization (WMO). These include
new elements that define temporal characteristics of data sets:
beginDateTime, endDateTime, and dataFrequency. Further information on
the WMO Community Core Metadata Profile can be found at
<http://www.wmo.int/web/www/WDM/Metadata/>
[documents](http://www.wmo.int/web/www/WDM/Metadata/documents.html).html.

Requirement 3.2.4 Further extensions to the FGDC and ISO standards are
allowed. Therefore, the CLASS metadata model must be able to accommodate
extensions.

Requirement 3.2.5 CLASS shall accept, manage and deliver metadata using
the guidelines established by the FGDC, augmented by applicable
supplemental profiles.

**Metadata Content**

Requirement 3.2.6 Dataset specific metadata requirements shall be
included in agreements between CLASS and the data providers. These
agreements should follow the form and intent of Submission Agreements
described in the OAIS reference model (See requirement 5.1).

Requirement 3.2.7 CLASS shall support recording of all processing steps
applied to the data and recording of those processing steps in the
lineage sections of the appropriate metadata records.

Requirement 3.2.8 Sensor Data Records, Environmental Data Records and
Climate Data Records and their associated processing algorithms and
external ancillary data should be documented in a consistent and
accessible manner. All metadata and processing procedures shall be
included to the extent that retrospective users should be able to
successfully repeat the production process.

Requirement 3.2.9 CLASS metadata shall include a flag that indicates
overall dataset quality, so that "bad" datasets can be identified.

Requirement 3.2.10 CLASS metadata shall include a set of controlled
vocabularies for items such as keywords, entities and attributes, and
units. To the extent possible, these vocabularies shall be consistent
with those maintained by related groups (i.e. GCMD, WMO, UCAR).

Requirement 3.2.11 ISO 19115 specifies a minimum set of elements needed
to make up an ISO conformant metadata record, including elements
classified as mandatory (M) and mandatory under certain conditions (C).
These "core" metadata elements are generally those required to identify
a dataset for the purpose of data discovery. All metadata profiles based
on ISO 19115 must include the core elements listed in the table below.
Therefore, these elements should be mandatory for CLASS.

(M: mandatory, C: mandatory if applicable)

  -------------- ---------------- --------------------------------------------
  **Element**    **Obligation**   **ISO Field/Class Name and hierarchy**

                                   

  Dataset title  M                MD_Metadata \>
                                  MD_DataIdentification.citation \>
                                  CI_Citation.title

  Dataset topic  M                MD_Metadata \>
  category                        MD_DataIdentification.topicCategory

  Abstract       M                MD_Metadata \>
  describing the                  MD_DataIdentification.abstract
  dataset                         

  Dataset        M                MD_Metadata \>
  reference date                  MD_DataIdentification.citation \>
                                  CI_Citation.date

  Dataset        M                MD_Metadata \>
  language                        MD_DataIdentification.language

  Metadata point M                MD_Metadata.contact \> CI_ResponsibleParty
  of contact                      

  Metadata date  M                MD_Metadata.dateStamp
  stamp                           

  Dataset        C                MD_Metadata \>
  character set                   MD_DataIdentification.characterSet

  Geographic     C                MD_Metadata \> MD_DataIdentification.extent
  location of                     \> EX_Extent \> EX_GeographicExtent \>
  the dataset                     EX_GeographicBoundingBox or
  (by four                        EX_GeographicDescription
  coordinates or                  
  by geographic                   
  identifier)                     

  Metadata       C                MD_Metadata.language
  language                        

  Metadata       C                MD_Metadata.characterSet
  character set                   
  -------------- ---------------- --------------------------------------------

**Metadata Management**

Requirement 3.2.12 CLASS shall include a metadata management system
(MMS) that supports an interactive Web interface that allows data
providers and CLASS staff to create and manage CLASS metadata.

Requirement 3.2.13 The MMS shall include mechanisms to generate,
validate, approve and maintain metadata.

Requirement 3.2.14 The MMS shall provide a capability to generate
metadata records from self-describing data sources in which metadata and
data have been integrated.

Requirement 3.2.15 The MMS shall provide mechanisms to extend metadata.

Requirement 3.2.16 The MMS shall provide the capability for data
providers to manage their metadata within a local system or through a
centralized system via a Web browser interface. It shall not require the
data provider to maintain duplicate copies of metadata in two or more
systems.

Requirement 3.2.17 The MMS shall include mechanisms to facilitate the
generation of metadata as close as possible to the collection and/or
generation of the source data.

Requirement 3.2.18 The MMS shall provide the capability to deliver
metadata along with data.

Requirement 3.2.18.1 The MMS shall provide the capability to generate
metadata to be consistent with data aggregates or subsets.

Requirement 3.2.18.2 The MMS shall allow users to specify the level of
detail in the metadata to be delivered (e.g. none, summary, standard or
comprehensive).

Requirement 3.2.18.3 The MMS should allow users to specify if they would
like metadata delivered as human-readable text, or as FGDC CSDGM or ISO
19115-compliant XML.

Requirement 3.2.19 The MMS shall allow users to submit metadata or
annotate CLASS metadata to include pointers to additional metadata that
the users might hold.

Requirement 3.2.20 The MMS shall provide support for parent/child
metadata with inheritance.

Requirement 3.2.21 The MMS shall include an automated maintenance
capability for checking URL links and any additional information within
metadata records.

Requirement 3.2.22 The MMS shall provide automated tools for version and
configuration management of metadata.

Requirement 3.2.23 The MMS shall provide a mechanism to access existing
metadata servers to harvest metadata compliant with the FGDC CSDGM or
ISO 19115.

Requirement 3.2.24 The MMS shall provide a mechanism to provide CLASS
metadata to other catalogs in either FGDC CSDGM or ISO 19115-compliant
XML.

Requirement 3.2.25 The MMS shall provide interactive or automated
(Application Program Interface (API)) access to all metadata items that
can be used for data discovery and data quality assessment.

Requirement 3.2.26 CLASS shall provide a mechanism to ensure that
metadata used for data discovery are up to date, consistent, and
understandable to its Designated Community.

### Interactive Web Interface

CLASS currently provides access to customers through a Web browser
interface. However, the current interface is too dataset-centric and
should be more metadata-centric. That is, it should present an interface
that allows users to locate datasets without their needing to specify a
particular instrument or sensor. The search interface should allow users
to search via specific field values or through free text (albeit with
controlled vocabularies). Users should be able to specify field values
through pull-down menus or, where appropriate, interactive maps and
gazetteers of geopolitical and scientific features.

Development and maintenance of multiple interfaces should be considered
to best meet the needs of the diverse customer base. While novice users
would be best served through extensive menus and forms, expert users
might prefer a terse command line interface or computer-to-computer
interfaces.

The search system should be extensible to support searches on any
metadata maintained by CLASS. It should offer customers the capability
to identify special attributes of the images they need. For example,
customers could identify cloud-free data sets by examining browse
images, but this can be very time consuming. Instead, they should be
able to search for only images that are cloud free.

The vast majority of NOAA data and related environmental observations
include a geospatial component. The interactive and automated searches
must both include the capability to take advantage of spatial
relationships between observations and other geospatial data. Examples
of these capabilities include:

-   Find satellite orbits that intersect a NEXRAD Station, a National
    Weather Service Forecast Zone, a River, a Congressional District or
    a Hurricane Track.

-   Find intersections between observations from sensors on two
    different satellites.

-   Find NOAA datasets that intersect satellite observations using
    metadata records, link to metadata repository for more information
    about the datasets.

-   Find in-situ NOAA observatories and observations that are included
    in satellite scenes, link to observing system and observatory
    information and data. This includes observations made at points as
    well as along lines or tracks.

-   Find multi-dimensional grid subsets that intersect a given satellite
    orbit.

Requirement 3.3 CLASS shall include an interactive Web browser interface
that allows customers to discovery, order, or download CLASS metadata,
data, products, documentation and software tools.

Requirement 3.3.1 The interactive interface shall provide information on
requirements for CLASS data providers.

Requirement 3.3.2 The interactive interface shall support the most
popular Web browsers, sufficient to reach at least 98% of its users.
CLASS shall annually monitor the most common browsers to ensure it keeps
abreast of any significant changes in the browser market.

Requirement 3.3.3 The interactive user interface shall be simple and
intuitive for the designated user community (technically literate
users).

Requirement 3.3.10 The interactive interface shall provide for customers
to "register" so that they can track their orders and save their search
preferences. (See requirement 3.5.7).

Requirement 3.3.5 The interactive interface shall conform to all Federal
guidelines on Internet presence.

Requirement 3.3.6 The interactive interface shall be Section 508
compliant (see <http://www.section508.gov/>).

Requirement 3.3.7 The interactive interface shall provide access to any
required policy statements and legal disclaimers.

Requirement 3.3.8 The interactive interface shall provide access to
frequently asked questions (FAQ) and their answers.

Requirement 3.3.9 The interactive interface shall provide access to
on-line documentation on itself, and on metadata and data managed by
CLASS.

Requirement 3.3.10 The interactive interface shall provide a "Help"
function, including a means to submit questions to CLASS staff.

Requirement 3.3.11 There shall be simplified versions of the interactive
interface suitable for incorporation into non-CLASS Web sites for
purposes of offering the capability to search CLASS data.

### Automated Machine Interface

Requirement 3.4 CLASS shall include a mechanism to respond to computer
requests for information concerning datasets from a system outside
CLASS.

Requirement 3.4.1 CLASS should provide access to these automated
requests through a Web-services interface to its metadata, which would
return metadata in a standard machine-readable form (e.g. FGDC-compliant
XML).

Requirement 3.4.2 CLASS shall provide access to its metadata via the
Z39.50 protocol and should also comply with the OGC Catalog Services
Specification.

### Search/Query System

Requirement 3.5 CLASS shall include a search system that searches
discovery-level metadata for dataset records that meet user-defined
criteria.

Requirement 3.5.1 All datasets managed by CLASS should be accessible to
the search system through every stage of their processing. That is, a
customer should be able to locate and request datasets as level 1, level
2, etc.

Requirement 3.5.2 The search system shall provide automated machine
access to any/all metadata fields through the OGC Catalog Interface (see
<http://www.opengis.org/specs/?page=specs>). This includes bindings with
Z39.50, CORBA and HTTP protocols.

Requirement 3.5.3 The search system shall provide full support for Open
GIS Web Map/Feature/Coverage searches.

Requirement 3.5.4 The search system shall be accessible via the
interactive Web interface and through this interface shall provide the
following types of searches:

Requirement 3.5.4.1 Field-specific searches with pull down menus or
interactive maps

Requirement 3.5.4.1.1 Search by parameter

Requirement 3.5.4.1.2 Search by time or time range, including the
capability to request only those data sets that include more than a
user-specified minimum time span.

Requirement 3.5.4.1.3 Search by area using interactive maps or
user-specified coordinates, including the capability to request only
those data sets that overlap more than a user-specified percentage of
the search area.

Requirement 3.5.4.1.4 Search by named or user supplied geospatial
feature (polygon, line, point)

Requirement 3.5.4.1.5 Search by data resolution

Requirement 3.5.4.1.6 Search by data frequency

Requirement 3.5.4.1.7 Search by special attributes of the data. For
example, search for only satellite images that are cloud free or radar
images that include more than clear air returns.

Requirement 3.5.4.2 Free text searches with single or multiple words

Requirement 3.5.4.2.1 CLASS shall support Boolean operator keywords used
in multiple word searches

Requirement 3.5.4.3 Command-line searches for expert users

Requirement 3.5.5 The interactive search system should be dynamic. That
is, it should be able to present different search options depending upon
the types of data or product that a customer has selected. Example 1:
after selecting "radar data" the additional option of selecting
individual sites would be presented. Example 2: after selecting model
forecasts the additional option of specifying a verification time would
be presented.

Requirement 3.5.6 The interactive search system should support iterative
refinement of searches.

Requirement 3.5.7 The interactive search system should allow registered
customers to save their search criteria for future reuse.

Requirement 3.5.8 The search system should be extensible to support
searches on any metadata maintained by CLASS.

Requirement 3.5.9 For each data set found as a result of a search, CLASS
shall return the dataset title and discovery-level metadata including an
abstract and spatial and temporal coverage. CLASS shall return a pointer
to all of the additional metadata available for each data set.

Requirement 3.5.9.1 For interactive searches the discovery-level and
additional metadata shall be presented in an easily understood
human-readable form.

Requirement 3.5.9.2 For automated machine access the results and
additional metadata shall be presented in FGDC CSDGM or ISO
19115-compliant XML.

Requirement 3.5.10 Datasets that are known to contain errors (usually
due to degraded instrument performance) should be obviously and visibly
flagged and users should be required to acknowledge this warning before
being able to order or retrieve these data. A possible mechanism would
be to include a "stoplight" indication of overall dataset quality: green
for no known problems, yellow for minor errors and red for known
significant errors.

Requirement 3.5.11 CLASS shall provide a basic browsing and
visualization capability that extends across the full breadth of CLASS
data. The browsing capability should provide geolocated and
time-referenced graphics and images that can be displayed by standard
Web browsers. These images should be suitable for the evaluation of
CLASS data.

Requirement 3.5.11.1 Browse images shall be available through both the
interactive and automated machine interfaces.

## Data and Product Delivery

Obviously, at the most basic level, CLASS must be able to deliver data
and products to its customers. These data and products should be
accompanied by complete OAIS Packaging Information that describes the
file names, directory structure, etc. For CLASS to fully satisfy its
users it must also provide powerful and flexible tools that allow users
to tailor output to their own needs and circumstances.

### Data/Product Selection

The CLASS data selection system should support:

-   Subsetting via spatial, temporal, or parameter criteria

-   Aggregation of multiple files into higher dimensional arrays

-   Statistical aggregation (mean, standard deviation, etc)

-   Format translation

Requirement 3.6 CLASS shall include a data/product selection system

Requirement 3.6.1 CLASS shall provide the capability for customers to
examine metadata and browse images resulting from data discovery and
then choose sets of interest for delivery.

Requirement 3.6.2 CLASS shall allow customers the ability to select from
the items returned from the search and/or create subsets of the returned
items.

Requirement 3.6.2.1 The CLASS data selection system shall support
generation of dataset subsets via spatial, temporal, or parameter
criteria.

Requirement 3.6.2.2 The CLASS data selection system shall support
aggregation of multiple files within a dataset into higher dimensional
arrays.

Requirement 3.6.2.3 The CLASS data selection system should support
statistical aggregation of data within a dataset (e.g. compute mean,
standard deviation, etc)

### Data/Product Translation

To be effectively used by customers, CLASS data and products should be
delivered in a format that the customer is familiar with and for which
he/she already has software tools. Unfortunately, even within a single
community and application several choices are often available. For
example, for multi-dimensional gridded data (e.g. from models) netCDF
and GRIB are widely used. HDF and HDF5 are also sometimes used for this
purpose but more commonly used for raster data.

Since no single standard is accepted or can meet the needs of all of
CLASS's customers, CLASS should provide the capability for customers to
receive data in any one of a few standard formats. Different formats are
appropriate for different types of data so the available choices must
depend upon the type of data or product needed. Each of the formats
specified below is described in Appendix 3.

Requirement 3.7 CLASS shall offer users capabilities and/or tools to
enhance the utility of data that it provides

Requirement 3.7.1 CLASS should allow users to select the format of the
data/products they will receive. The following options should be
supported:

  ------------------------ ----------------------------------------------
  **Data/Product Type**    **Delivery Formats**

  Satellite data           netCDF, HDF, HDF5, HDF-EOS, native formats,
                           GeoTIFF, BUFR

  Satellite imagery        netCDF, JPEG, GIF, GeoTIFF

  Satellite image loops    Animated GIF, MPEG

  Gridded model data       netCDF, GRIB, HDF, HDF5, GeoTIFF

  Radar data               UF, netCDF, WSR88D native formats

  Radar imagery            netCDF, JPEG, GIF, GeoTIFF

  Radar image loops        Animated GIF, MPEG
  ------------------------ ----------------------------------------------

Requirement 3.7.2 CLASS shall provide pointers to software tools that
can be used to read any of the data formats that can be delivered by
CLASS.

Requirement 3.7.3 CLASS should support delivery of compressed datasets,
as this will allow easier user access to very large datasets with no
significant loss of information. Hyperspectral measurements from future
satellite constellations, such as METOP, NPP, NPOESS and GOES-R will
significantly increase the volume of satellite data for a given spatial
coverage. These enormous datasets will inhibit the utilization of the
data by a number of CLASS users unless they are available in compressed
form. Principal component analysis has been used experimentally to
achieve a compression rate of 50 times for EOS Atmospheric InfraRed
Sounder (AIRS), thereby reducing the volume of an orbit of AIRS to
approximately 50 megabytes from the original 2.5 gigabytes. Even though
the compression is lossy, the missing component has been demonstrated to
be nearly all noise.

### Data/Product Delivery

Requirement 3.8 CLASS shall provide the capability to deliver data and
products to customers

#### Delivery via the Internet

#### [Standard Protocols]{.underline} {#standard-protocols .unnumbered}

CLASS should be able to deliver data and products to customers over the
Internet using standard protocols such as FTP and HTTP.

CLASS should also seriously investigate modern alternatives that have
been developed to serve the peer-to-peer file sharing community. One
example is Gnutella, which offers many advantages over FTP such as
support for automatic restarts of interrupted downloads and parallel
downloading in slices from multiple servers.

Requirement 3.8.1 CLASS shall be able to deliver data and products to
customers over the Internet using standard protocols, including FTP and
HTTP.

#### [Standard APIs]{.underline} {#standard-apis .unnumbered}

Interoperability between Earth observation data systems depends
crucially on program interfaces, typically described through standard
service definitions. Such service definitions precisely specify the
syntax and semantics of all data elements exchanged and fully describe
how systems interact at the interface. At present, several standard APIs
are available and no single standard has emerged as the clear leader.
However, CLASS should use one or more of the most common APIs that are
currently in use within its community. These include OPeNDAP, OGC Web
Map/Feature/Coverage Server, and SOAP/WSDL, which are described in
Appendix 3. CLASS should keep abreast of developments in this area and
adapt standards as they become widely accepted.

Requirement 3.8.2 CLASS shall be able to select and deliver data and
products to customers over the Internet using community-standard APIs.

Requirement 3.8.2.1 CLASS shall provide access to data via OPeNDAP
(<http://opendap.org/>)

Requirement 3.8.2.2 CLASS shall provide access to maps and images via
the OGC Web Map Server protocol
(<http://www.opengis.org/docs/01-068r2.pdf>)

Requirement 3.8.2.3 CLASS shall provide access to data and products via
Simple Object Access Protocol (SOAP) and Web Services Description
Language (WSDL), *[if they are widely accepted and adopted by the CLASS
user community.]{.underline}*

#### Delivery via Removable Media

Although the growth of Internet coverage and bandwidth allows CLASS to
distribute an increasing volume of data on-line, for the foreseeable
future some customers will require off-line delivery of media. As an
example, recently a customer discussed setting up a subscription that
would require delivery of about 140GB/day. Given the large volume of
data they preferred to receive it via removable media rather than
on-line.

Requirement 3.8.3 CLASS shall provide the capability for customers to
order data be delivered on removable media. A variety of up to date
media should be available including cartridge tapes, 8mm tapes, CD-ROM
and DVD-ROM.

### Documentation

Data and products received from CLASS must be independently
understandable**.** That is, they should have sufficient documentation
to allow the information to be understood and used without needing the
assistance of the experts who produced them. The amount of metadata
required to adequately characterize the data depends upon the user and
purpose for which the data will be used. As an illustration, consider
the following two examples.

1)  Robert would like to include a visible satellite image of a
    hurricane in his term paper. Assuming that the image is available in
    a standard format that can be displayed by a Web browser, the only
    metadata that he needs are the name of the hurricane and the date of
    the image.

2)  Susan is studying the impact of tropical storms on sea surface
    temperature. She would like computed sea surface temperatures of the
    Gulf of Mexico immediately before and after the passage of a
    hurricane. Along with the images, she needs detailed information on
    the date and time of each, the source satellite, the spectral bands
    and algorithm used to compute the temperatures, the calibration
    coefficients that have been applied, and the expected error range.

For CLASS to best satisfy the needs of its users, it must be able to
provide a different level of documentation and metadata to different
users. Therefore, CLASS must collect and maintain metadata detailed
enough to meet the needs of the most demanding users that can be
imagined, but should allow users to retrieve only the level of metadata
that they require.

Requirement 3.8.4 CLASS shall provide the capability to deliver metadata
along with data in accordance with customer requirements (see
Requirement 3.2.17.2)

## Subscriptions

At times customers might prefer to receive a given product on a regular
schedule, i.e. set up a subscription. In addition to the normal criteria
used to select a dataset, the customer should also be able to define the
time range when the subscription is valid (e.g. from May 1 to September
30) and the frequency at which data will be delivered (e.g. daily). Once
defined, the subscription could either push the data to the user
automatically, or notify the user that the data is available. CLASS
should support both approaches.

CLASS should manage subscriptions specific to an individual customer
rather than as a generic subscription for all customers who are
interested in a particular dataset. In that way, customers will only
receive products that meet their requirements and will not need to
evaluate the product to determine its suitability whenever a
notification is received. Furthermore, individually managed
subscriptions would allow customers to receive products tailored to
their needs (e.g. subsets or data delivered in a specific format).

Requirement 3.9 CLASS shall allow customers to set up a subscription to
receive a given product on a regular schedule.

Requirement 3.9.1 The subscription capability shall allow customers to
specify any of the criteria that can be used to select a dataset via the
discovery process.

Requirement 3.9.2 The subscription capability shall allow customers to
define the time range when the subscription is valid

Requirement 3.9.3 The subscription capability shall allow customers to
define the frequency at which data will be delivered.

Requirement 3.9.4 The subscription capability should support any of the
data selection or transformation functions that are available for
non-subscription orders.

Requirement 3.9.5 CLASS shall support, at the discretion of customers,
either automatically pushing subscribed products to the customer or
sending a notification when the data are available (and where and how to
get it).

Requirement 3.9.6 CLASS should manage subscriptions at a level specific
to an individual customer.

## Customer Feedback

Requirement 3.10 It is essential that customers be able to provide
feedback on their experience with CLASS. CLASS should provide the
opportunity for customers to volunteer opinions via e-mail, or via an
optional customer survey form in which they could reply to specific
questions regarding the selection, delivery and use of CLASS data.

Requirement 3.11 CLASS staff should monitor feedback received through
e-mail and CLASS should develop procedures and mechanisms for creating,
updating and reviewing completed survey forms.

# Requirements for Archive and Records Management

The National Archives and Records Administration (NARA) oversees the
management of all Federal records. All Federal agencies are required to
manage records in accordance with the NARA regulations, as codified in
36 Code of Federal Regulations, Chapter 12. Particularly important for
CLASS is NARA Federal Regulations 36 CFR, Chapter 12, Subchapter B, Part
1234 *- Electronic Records Management*
(<http://www.archives.gov/about_us/regulations/part_1234.html>). Subpart
C \-- *Standards for the Creation, Use, Preservation, and Disposition of
Electronic Records*.

In general, NARA regulations stipulate that Agencies shall:

-   Maintain adequate and up-to-date technical documentation for each
    electronic information system that produces, uses, or stores data
    files

-   Provide for backup and recovery of records to protect against
    information loss

-   Ensure that information is not lost because of changing technology
    or deterioration by converting storage media to provide
    compatibility with the agency\'s current hardware and software

-   Maintain magnetic computer tapes in accordance with specific NARA
    guidelines (For example: "Agencies shall copy permanent or
    unscheduled data on magnetic tapes before the tapes are 10 years old
    onto tested and verified new tapes.")

-   Periodically review the contents of archives for disposition in
    conjunction with data center NARA guidelines.

Requirement 4.1 CLASS must support management of its data in accordance
with NARA regulations

# Producer Requirements

## Submission Agreements

The CCSDS Reference Model for an OAIS identifies the need for agreements
between data providers and the archive system. These are termed
Submission Agreements and are defined as: "*agreements reached between
an OAIS and the Producers that specify a data model for the Data
Submission Session. This data model identifies format/contents and the
logical constructs used by the Producer and how they are represented on
each media delivery or in a telecommunication session."*

The reference model also describes the interactions that occur between
the data providers and the OAIS -- subsequently referred as the Archive
in this document- (Section 2.3.2 of the Reference Model) and the role of
the Submission Agreement in those interactions:

-   The first contact between the Archive and the data provider is a
    request that the Archive preserve the data products created by the
    data provider. The Archive, the data provider or Management may
    initiate this contact

-   The data provider establishes a Submission Agreement with the
    Archive, which identifies the Submission Information Packets (SIPs)
    to be submitted and may span any length of time for this submission.

Within the Submission Agreement, one or more Data Submission Sessions
are specified. The Data Submission Session content is based on a data
model negotiated between the Archive and the data provider in the
Submission Agreement. This data model identifies the logical components
of the Submission Information Package (which includes Content
information, Preservation description information, Packaging
information, and Descriptive information) that are to be provided and
how (and whether) they are represented in each Data Submission Session.
A brief explanation of the components of information packets is given in
Appendix 6 of this report and a detailed explanation is provided in
section 4.2.1.4 of the OAIS Reference Model.

Requirement 5.1 CLASS should establish Submission Agreements with all of
its data providers. Several steps are required to create a Submission
Agreement and proposal for this process, is provided in Appendix 1.

Requirement 5.2 Datasets that are known to contain bad data present a
particular problem for CLASS. Submission Agreements with CLASS data
providers should establish one or more automatic mechanisms for
bi-directional communication regarding how to address issues concerning
bad datasets.

Requirement 5.3 When negotiating the details of a Submission Agreement,
CLASS should strive to establish a long-term relationship with the data
provider to ensure that they will provide as much support as possible
for their datasets.

# Management Requirements

## Configuration Management

A commitment to effective hardware and software configuration management
practices is essential for any significant effort in information
technology. Definition of these practices for CLASS is not within the
scope of this document. However, this document describes archive and
access system requirements from the perspective of CLASS customers. In
this regard:

Requirement 6.1 CLASS should include the capability to accept requests
for enhancements from customers.

Requirement 6.2 To reduce the likelihood of duplicative enhancement
requests, CLASS should allow customers to view a summary of planned or
proposed upgrades.

Requirement 6.3 CLASS should also, if possible, allow customers to
ascertain the general status of pending requests (e.g. under review, to
be considered for implementation in the future, accepted and scheduled
for implementation, etc.).

## Usage Metrics

Requirement 6.4 CLASS should monitor use of its interactive and
automated interfaces to detect patterns of use that could suggest
requirements for new options or capabilities. Within its log of usage
statistics CLASS should record at least the following:

a.  Data access mechanism (interactive or automated service)

b.  Metadata fields used in the selection

c.  Boolean operators used (if supported)

d.  Was additional metadata requested for review and, if so, the level
    of detail wanted?

e.  Browse or summary products requested

f.  Was data requested for delivery? If so,

    -   Delivery mechanism (e.g. FTP, OPeNDAP, etc.)

    -   Data subset selection criteria

    -   Data transformation/reformatting

    -   Data volume transferred and number of files

    -   IP address (or at lease domain name) of the customer

    -   Level of detail of metadata requested for delivery with the data
        and, if so, the metadata format requested (e.g. human readable,
        XML)

## Organization Priorities

One of NOAA's cross-cutting priorities is to develop an Integrated
Global Environmental Observation and Data Management System. To reach
this goal and to enhance efficiency within the organization, it is
essential that the CLASS project coordinate its actions with related
data management activities both within and outside of NOAA, (e.g.
activities listed in Appendix 2.) It should investigate what other
agencies are doing, benefit from their experiences and adopt their
successful practices to ensure CLASS does not duplicate their work.

Requirement 6.5 The CLASS project should formally and informally reach
out and work with related data management activities, both within and
outside of NOAA to take advantage of their experience, minimize
duplication of effort and ensure maximum compatibility between systems.
Special attention should be given to existing systems that provide
access to model and radar data to ensure CLASS makes maximum use of
their experience, capabilities and software.

# Priorities for Action

This document describes an extensive and diverse set of requirements for
further CLASS development that have been explicitly defined by CLASS
customers or are implicitly required for effective and responsible data
stewardship. The ARWG has not attempted to individually assign
priorities to the detailed requirements listed in the appendices. Rather
the ARWG recommends that further development of CLASS proceed with the
following general activities, which are listed in order of priority. All
activities should be pursued in accordance with the requirements defined
within this document.

a.  Clearly define a prioritized list of large-array data sets to be
    included in CLASS and a timetable for their implementation. Engage
    CLASS users in this process.

b.  Establish a CLASS User Working Group and set up an Internet forum.

c.  Design a comprehensive metadata model for CLASS.

d.  Develop and implement a metadata management system (MMS) for CLASS
    metadata.

e.  Establish formal Submission Agreements with all CLASS data
    providers, which include agreement on which metadata elements are to
    be required.

f.  Transform and load existing CLASS metadata into the MMS.

g.  Design, develop and implement a new CLASS interactive interface
    that:

-   utilizes metadata for data discovery

-   provides a mechanism to solicit customer feedback

-   collects usage metrics

h.  Implement an automatic machine interface for data/product discovery
    that:

-   utilizes metadata for data discovery

-   collects usage metrics

-   supports the OGC Catalog Interface Specification

i.  Implement one or more APIs for data access and delivery.

j.  Collect all required additional metadata for CLASS datasets and load
    into the MMS.

k.  Add capabilities for data/product selection and transformation to
    the interactive and machine interfaces.

l.  Enhance the subscription service to support capabilities for
    data/product selection and transformation.

m.  Enhance the machine interface for data access and delivery to serve
    as a complete Web-enabled service.

**Appendix 1**

**Procedures for Negotiation of a Submission Agreement**

Negotiation between a Data Producer and an Archive leading to a
Submission Agreement should, to the maximum extent possible, follow a
standard set of procedures. The diversity of data and data providers
being considered in CLASS and our lack of experience with submission
agreements suggest that it may take some time for this set of procedures
to emerge and that the procedures will always need significant
flexibility. Two recent documents discuss these procedures in some
detail: the Consultative Committee for Space Data Systems (CCSDS)
Producer-Archive Interface Methodology Abstract Standard (CCSDS
651.0-B-1), May 2004 and A Procedure for Developing Engineering
Requirements by Bruce R. Barkstrom. This Appendix paraphrases the
procedure described by Barkstrom as an initial strawperson proposal for
procedures to be used by CLASS.

Most of this activity is conducted by a Data Producer Representative and
an Archive Representative. The understanding reached by the parties at
each stage is marked by completion of a document.

There are several sub-activities that lead to the Finalized Submission
Agreement with accompanying documents that constitute the sub-sections
of the final agreement. For each of these sub-phases, the concluding
activity is to store a copy of the document that marks the end of the
sub-phase into a directory in the Archive that will hold the history of
the negotiation. That way, when the Submission Agreement needs revision,
the previous version of the history can be extracted and used to guide
the revision.

***1. Initial Contact Between Data Producer and Archive***

The OAIS Reference Model notes, "The first contact between the OAIS and
the Producer is a request that the OAIS preserve the data products
created by the Producer. This contact may be initiated by the OAIS, the
Producer or Management." The OAIS Reference Model defines several types
of Information Package, one of which is the Submission Information
Package (SIP). A detailed outline of SIP\'s and other documentation is
provided in this section and in other documents.

To assist the Data Producer in preparing for the negotiation process, it
is helpful to have a draft version of the Submission Agreement
Negotiation Schedule that can be revised as the negotiation proceeds.

This initial contact should be kept simple. The steps involved are as
follows:

a.  The Archive should disclose to the Data Producer the process of
    negotiating a Submission Agreement, including the Schedule for the
    negotiation.

b.  During the discussion, the Archive Representative should keep
    minutes of the meeting that can be included in the Notes directory
    after that has been set up in the Archive\'s Submission Agreement
    History.

c.  At the conclusion of this contact, the Archive should open a
    Submission Agreement History directory with subdirectories
    associated with each type of documentation.

***2. Obtain Collection Description***

The Producer Collection Description is intended to be a brief
description, created by a Data Producer of the content and expected use
of a collection to be accepted by the Archive. It should include
preliminary information regarding the number and size of files, as well
as the Data Producer\'s expectations regarding the Designated User
Community.

To help the Data Producer Representative document this view, the Archive
Representative should ask the following questions:

-   What kinds of data will be stored in the Archive\'s collection?

-   What are the (homogeneous) sub-collections within the Producer\'s
    overall collection?

-   About how many files will be in each sub-collection?

-   About how big are the files in each sub-collection?

-   How many Submission Sessions are there likely to be?

-   How long will it take to ingest the data in a Submission Session?

-   What kinds of Quality Assurance work is expected on each file coming
    in?

-   What kind of metadata and provenance information will be supplied
    with each file?

-   What services does the Data Provider expect, need, or require for
    the collection?

-   What user groups are there likely to be for the data in the
    collection?

-   How many users are there likely to be in each user group?

-   Is there any experience to help quantify how many files each user
    group is likely to order?

-   Are the users likely to need or want data on media?

After the session ends, the Archive should do the following:

a.  Record the minutes of the meeting in the notes section of the
    Submission Agreement History.

b.  Set up a deadline for receiving a text version of the answers to
    these questions - in so far as the Data Producer may be able to
    provide it.

c.  When the text arrives, load it into the Submission Agreement History

***3. Conduct Collection Valuation and Risk Assessment***

Conducting a collection valuation and risk assessment is a critical
element in making sensible design decisions for an Archive. The steps
involved in this activity follow:

**Risk Assessment**

a.  Risk identification - define a list of project-specific risk items.

b.  Risk analysis - assess the loss probability and loss magnitude for
    each identified risk item, as well as compound risk-item
    interactions.

c.  Risk prioritization - produce a ranked ordering of the risk items
    identified and analyzed.

**Risk Control**

d.  Risk-management planning - to address how to deal with each risk
    item.

e.  Risk resolution - produce a situation in which the risk items are
    eliminated or otherwise resolved.

f.  Risk monitoring - after the plans and resolution activities have
    begun, track the project\'s progress towards resolving its risk
    items and taking appropriate corrective actions.

**Finalizing the Initial Session**:

g.  Record the Data Producer\'s valuation and concepts regarding the
    possible loss impacts.

h.  The Archive should supply an independent assessment of the threats.

i.  The Archive should discuss this assessment with the Data Producer
    and resolve any issues.

j.  The final version of the Valuation and Risk Assessment should be
    stored in the Submission Agreement History directory.

***4. Quantify User Access Model***

Quantifying the User Access Model is critical for engineering a number
of Archive capacities - storage throughput, storage device data access
patterns and optimization, user search strategy and Archive metadata
design, as well as distribution cache sizes and data retention period.

The large-scale steps in this process are

a.  Identify the Value Proposition for the user in the proposed
    collection identified by the Data Producer.

b.  Segment the Market for the various value propositions identified.

c.  Quantify the expected potential population that might use the
    collection contents.

d.  Forecast the Market and Income Stream, possibly using the
    Information Diffusion model commonly used by market researchers.

After the Data Producer\'s Representative has performed a preliminary
form of this analysis, the following steps are expected:

e.  The Data Producer should provide a draft model to the Archive for
    joint work in refining the estimates.

f.  When the Data Producer and the Archive agree on the document, it
    should be inserted into the Submission Agreement History.

***5. Conduct Engineering Analysis of Data Producer Submissions***

The Engineering and Cost Analysis is the in which the Data Producer
provides critical requirements and the Archive provides an estimate of
the cost of those requirements.

The Engineering and Cost Analysis includes a number of steps to arrive
at a preliminary cost model:

a.  Obtain the Data Ingest Profile from the Data Producer and translate
    into Required Storage as a function of time.

b.  Obtain Average Peak Throughput Requirements from the Data Producer
    and analyze the required data transfer load within the Archive.

c.  Obtain the required computational services from the Data Producer
    and analyze the resulting computational load to quantify the
    computational resources required for the collection.

d.  Obtain the Data Distribution Requirements consistent with the User
    Access Model and analyze the resulting network bandwidth, router
    throughput, and similar capacity requirements.

e.  The Archive should estimate the required personnel needs for meeting
    the operational burden imposed by the proposed collection.

f.  When the requirements have been quantified by the Data Producer and
    the Archive, that information should be used by the Archive to
    populate a Cost Model for the proposed resources required by the
    proposed collection.

g.  Minutes from meetings between the Data Producer\'s Representatives
    and the Archive\'s Representative should be inserted in the
    Submission Agreement History.

h.  When agreement is reached on the Requirements, Engineering Analysis,
    and Cost Model, these items should be documented with the agreed to
    Engineering Analysis, the Operations Scenario, and the Cost Model
    being inserted into the Submission Agreement History.

***6. Develop Service Delivery Agreement \[if needed\]***

The Delivery Service Agreement is a plan for delivering special
equipment or software that was not part of the Archive\'s pre-existing
complement of hardware or software.

***7. Define Producer Logical Namespace and Metadata Model***

Because of the large variability from Data Producer to Data Producer in
their \"data world view\", the Archive Representative must work with the
Data Producer Representative to define the appropriate Logical Namespace
Hierarchy and Logical Metadata Structures to use with the Data
Producer\'s data.

***8. Metadata Database and Database Access Interface***

Once the Logical Namespace Hierarchy and Logical Metadata Structure are
available, work can proceed on ensuring the Archive's Metadata Database
and Database Access Interface are sufficient to meet these requirements.
If not, the Archive Metadata Database must be expanded as necessary. It
is expected that the Logical Metadata Structure will identify the
appropriate tables into which the metadata values fit.

***9. Design Submission Schedule***

To design the Submission Schedule, preliminary versions of a number of
key data structures are needed: notably the kinds, numbers, and sizes of
the files to be submitted. It is also assumed that the Data Producer\'s
Representative knows about what the schedule for submitting these files
should be.

It is expected that the Data Producer\'s Representative will formalize
the submission in two ways:

a.  Define the content of the Submission Information Packages (SIP\'s) -
    identifying how the files, their associated metadata, and the
    provenance information will be assembled into the actual Information
    Packages. The Archive will use this information to decide whether
    the SIP is acceptable for inserting into the Archive.

b.  Define the schedule for the submission of the SIP\'s. The schedule
    needs to include a fairly specific date and time of submission, as
    well as the duration of the window within which to accept the SIP.
    The Archive will use this information to control the input ports and
    their accessibility by the Data Producer.

***10. Finalize Producer-Archive Submission Agreement***

Finalizing the Producer-Archive Submission Agreement is a brief review
session to ensure that all of the expected documentation is available,
followed by the Data Producer and the Archive signing the Submission
Agreement. This work is divided into five steps:

a.  Verify that the history of the work involved in setting up the Data
    Producer-Archive Submission Agreement is in place and up-to-date.

b.  Provide at least one Authorized Data Producer Representative, his
    (or her) user ID, a trust authority that can verify submissions, and
    one or more IP addresses that this Representative can use in
    submitting authorization information to the Archive.

c.  Verify that the allocation of Storage Capacity for the Data,
    Metadata, and Provenance History portions of the Archive Storage
    Partition that will be allocated to this Data Producer is acceptable
    to both parties of the agreement.

d.  Verify that both parties agree to the services identified in the
    Data Producer-Archive Submission Agreement.

e.  Obtain a signed copy of the Data Producer-Archive Submission
    Agreement that can be placed in the Archive\'s Submission Agreement
    History for this Data Producer.

**Nominal Documents Produced as a Result of a Successful
Archive-Producer Negotiation**

***Submission Agreement Negotiation Schedule***

The Submission Agreement Negotiation Schedule initially provides a
notional schedule for negotiating a Submission Agreement. The schedule
provides a list of major negotiation processes, an expected duration for
each process, and the documents that mark the end of each process.

***Producer Collection Description***

The Producer Collection Description is a brief description created by a
Data Producer of the content and expected use of a collection to be
accepted by the Archive.

***Producer Collection Valuation and Risk Assessment***

The Producer Collection Valuation and Risk Assessment contain several
components: Collection Inventory (in present state and after archival),
Collection Valuation, Threat Analysis, Loss Analysis, and Risk Reduction
Strategy Identification.

***User Access Model (Archive Business Plan)***

The User Access Model (or Business Plan provides a description of the
benefits the user may obtain from the use of the collection, quantifies
the Designated User Community, identifies key services for the Archive
to provide, provides an Availability Schedule (which should be
consistent with the Submission Schedule), User Access Adoption Model, an
Archive Support Model, and an Initial Cost Forecast.

***Producer Service and Capacity Plan***

The Producer Service and Capacity Plan flows from the Engineering
Analysis of the preceding components of the Submission Agreement. It
includes the Expected Storage Requirements Profile, the Service
Computation Requirements Profile, the IT Options Selection, and
Potential Cost Scenarios. This Plan may also include Hardware
Procurement and Installation Plans, Software Installation Plans,
Installation Procedures, and Maintenance Procedures.

***Logical Namespace Hierarchy***

The Logical Namespace Hierarchy is the specification of the directory
(or path) naming convention for the base archive element location. The
Logical Namespace Hierarchy should be described with a recursive XML
description.

***Logical Metadata Structure***

The Logical Metadata Structure describes the organization of the
metadata. Some metadata may be associated with collections, while other
metadata may be associated with the individual elements of the
collection. The design of the metadata database is derived from the
Logical Metadata Structure, but not identical with it.

***Metadata Database***

The Metadata Database document describes the database holding the
metadata for the Archive. The document should indicate any additional
elements or relationships that need to be added to the existing Archive
metadata database.

***Metadata Search Interface***

The Metadata Search Interface document describes the interface through
which Archive users can find and order data by using metadata or
directory searches.

***SIP Content Description***

A SIP Content Description identifies the components of a class of
Submission Information Packages (SIPs). For a replicable archive, the
SIP\'s are assumed to contain data, metadata, provenance information,
and unpacking instructions.

***SIP Delivery Schedule***

A SIP Delivery Schedule provides a list of SIP\'s, in which each SIP to
be submitted to the Archive has a name or identifier, an appropriate
(valid) IP address from which it will be submitted for ingest, and a
time window within which the Data Producer expects to deliver the SIP.

***Finalized Submission Agreement***

The Finalized Submission Agreement identifies that a Submission
Agreement is in place and has been accepted by the Data Producer and the
Archive. The Finalized Agreement references the most current versions of
all the subsidiary components of the Submission Agreement, and
identifies the conditions under with the agreement may be terminated and
the process for disposing of the Data Producer\'s items stored in the
Archive. The Agreement needs to provide at least one Authorized Data
Producer Representative, a trust authority that can verify submissions,
and one or more IP addresses that this Representative can use in
submitting authorization information to the Archive. The Finalized
Submission Agreement should include an agreed to allocation of Storage
Capacity for the Data, Metadata, and Provenance History portions of the
Archive Storage Partition that will be allocated to this Data Producer.
The Finalized Submission Agreement must include a list of the Archive
Services and their attributes. There must be a Finalized Submission
Agreement before further activities relating to Archive Submission can
commence.

**Appendix 2**

**Related Data Management Activities**

**Group on Earth Observations**

In July 2003, the United States hosted the [Earth Observation
Summit.](http://www.earthobservationsummit.gov/index.html) The summit
adopted a declaration that signified a political commitment toward the
development of a comprehensive, coordinated and sustained Earth
Observation System to collect and disseminate improved data,
information, and models to stakeholders and decision makers. To further
this goal, the Summit participants launched the intergovernmental ad hoc
Group on Earth Observations (GEO) to develop a 10-Year Implementation
Plan. Nine months later, in Tokyo, a second Summit was held and more
than 50 nations formally adopted the Framework Document for a ten-year
implementation plan for a Global Earth Observation System of Systems or
GEOSS. The plan itself will be presented at Earth Observation Summit III
in February 2005.

GEOSS is envisioned as a large national and international cooperative
effort to bring together existing and new hardware and software, and to
make it all compatible to supply data and information as efficiently and
inexpensively as possible. In accordance with the report of the GEO
Subgroup on Architecture, "GEOSS should be a comprehensive, coordinated,
and sustained system of systems, driven by user needs. It should be
flexible and evolvable in that it supports a broad range of
implementation options, allowing observing system developers to reflect
the best available understanding of the observation impacts and results,
and providing observing system developers the flexibility to use the
most appropriate technologies, observing approach, platform, and vantage
point. GEOSS should be innovative in its ability to incorporate new
technology, methods, and collaboration arrangements."

Furthermore the report states, "GEOSS should ensure that data and
products

-   Are observed, recorded and stored in clearly defined formats;

-   Have quality indications so that users can evaluate their value;

-   Are archived in defined formats and maintained as accessible data
    sets;

-   Have metadata information that enables search and retrieval by
    users;

-   Provide access to all required observations at national, regional
    and global scales."

**Interagency Working Group on Earth Observations**

Subsequent to the formation of GEO, the Interagency Working Group on
Earth Observations (IWGEO) was formed to develop a 10-year plan for
implementing the United States' components of the integrated Earth
Observation System. Most importantly for CLASS, among the roles and
responsibilities of the IWGEO, it is to develop the U.S. National Plan
for Earth Observation System, and coordinate with academia, the private
sector, and standards bodies on data quality and data management.
Further development of CLASS must be closely coordinated with this
activity.

**IOOS Data Management and Communications**

The U.S. marine science community has come together to plan, design and
implement an Integrated Ocean Observing System (IOOS). IOOS is
envisioned as a network of regional, national, and global systems that
rapidly and systematically acquires and disseminates data and data
products and will be the U.S. contribution to the international Global
Ocean Observing System.

The IOOS Data Management and Communications Steering Committee developed
a detailed implementation plan for the data management and
communications (DMAC) component of IOOS. The DMAC Plan provides a
comprehensive vision and strategy for integrating marine data streams
across disciplines, organizations, time scales and geographic locations.
The primary focus of the plan is on enhancing interoperability of
existing systems through development and/or adoption of standards and
protocols for metadata, data discovery, transport and on-line browse.
Although specific implementation details have not yet been defined, the
data and information system envisioned in the DMAC Plan is clearly
described.

DMAC intends to comply with all applicable federal, international and
community guidelines and standards. However, to ensure compatibility
between systems, further extension to and definition of some of these
standards is required.

DMAC and all of the other participants in GEOSS will require tools and
procedures to provide, manage and query metadata. Since DMAC expert
teams are pursuing a number of activities in this area, CLASS should
closely coordinate its metadata activities with these teams to ensure
that they work toward common goals whenever practicable.

**Existing NOAA Data Systems**

The NOAA data centers have been providing digital data to customers for
many years and have developed a number of data and information systems
to support this activity. These systems have, in general, attempted to
keep pace with advances in information technology and many have
developed significant customer bases. Some of these systems are
pertinent to CLASS in that they

-   Provide access to datasets included within CLASS's objectives,

-   Manage metadata relevant to CLASS,

-   Utilize emerging technology that could be used by CLASS, or

-   Utilize easy to use intuitive interfaces that could serve as models
    for future CLASS development.

[NOMADS]{.underline}

To address a growing need for remote access to high volume numerical
weather prediction and global climate models and data, the National
Climatic Data Center (NCDC) along with the National Centers for
Environmental Prediction (NCEP) and the Geophysical Fluid Dynamics
Laboratory, initiated the NOAA Operational Model Archive and
Distribution System (NOMADS) project. The NOMADS framework was also
developed to facilitate climate model and observational data
inter-comparisons.

The NOMADS framework is a distributed data system that promotes
combination of data sets between distant participants using open and
common server software and methodologies. Users can access model and
observational data and products in a flexible and efficient manner from
archives or in real-time over the Internet.

The framework of the server and software used to manipulate the archive
data base and distribute the real time data is OPeNDAP, which is a
protocol designed for the transport of scientific data subsets over the
Internet (see section 3.3).

NOMADS systems have been operational at NCEP and NCDC since August 2003,
serving approximately 4 million downloads of numerical weather
prediction model output per month. Both the National Weather Service and
the World Meteorological Organization (WMO) are currently valuating
NOMADS as a template for operational services within the U.S., and for
international use within the framework of WMO.

Additional information is available at:
<http://www.ncdc.noaa.gov/oa/model/model-resources.html>.

[NOAA Metadata Manager and Repository]{.underline}

The NOAA Metadata Manager and Repository (NMR) has been recently
developed through a cooperative effort by the NOAA National Data
Centers: the National Oceanographic Data Center (NODC), National
Geophysical Data Center (NGDC), and NCDC and the NOAA Coral Reef
Information System (CoRIS). These centers share the capabilities and
management responsibilities of this system. It was developed to provide
a single tool for NOAA data and metadata managers to create and manage
NOAA's metadata. The database model is based on the Federal Geographic
Data Committee (FGDC) metadata standard (described in section 3.1). The
browser-based interface was developed by a team from the National
Oceanographic Data Center (NODC) and the National Geophysical Data
Center (NGDC) using Java Servlets. The NMMR accepts structured
descriptive metadata in XML that is consistent with the FGDC metadata
standard. Descriptive metadata content can be edited, parsed, approved
and published using NMMR tools. Metadata from the NMMR for each archived
data set is used to populate the NOAA Server system (described below),
where it is accessible to the public.

The NMR interface allows access to all of the FGDC metadata fields.
Several task specific interfaces that include a subset of the FGDC
standard in a more specialized context have also been developed. The
comprehensive interface is designed for metadata managers that are
familiar with the FGDC standard. The task-specific interfaces facilitate
metadata collection from users that need to create standards compliant
metadata but are not familiar with the complete FGDC standard.

[NOAA Server]{.underline}

NOAA Server was conceived and developed in the mid 1990s. Its purpose
was to develop a framework and to implement a methodology, utilizing Web
technologies, which presents a unified view of NOAA's existing
nationally distributed environmental information databases. In addition
to defining a framework within which an intuitive and unified agency
environmental data and information server view could be established,
NOAA Server attempted to address the need for an information model and a
methodology for developing electronic inventories and catalog services.

As the project was implemented, it became clear that consistent metadata
was required to support both a unified view of NOAA data and discovery
of those data. Therefore, NOAA Server evolved from a unified interface
project into a metadata project. At present, it contains the only
collection of metadata collected from multiple NOAA Line Offices and the
only unified interface for searching metadata across multiple Line
Offices. NOAA Server also coordinates and exchanges descriptive metadata
with other federal clearinghouses, such as the NASA Global Change Master
Directory (GCMD) and the FGDC Geospatial Clearinghouse.

At its peak, NOAA Server linked together more than a dozen environmental
databases in many organizations within NOAA. Participating organizations
included elements of all the NOAA Line Offices. However, NOAA Server is
now moribund. There is no active support for the system, it has not
evolved beyond its state from the mid 1990s, and nearly all of the links
on the project information pages are obsolete and inoperable.

Further information is available at
<http://www.joss.ucar.edu/NOAAServer/>.

[NVDS]{.underline}

Developed in 1996, NOAA's National Virtual Data System (NVDS) was NOAA's
first e-Government system designed to simplify access to its data
holdings stored at three data centers. The project provides a single
system of access that allowed users to locate, access, and order data
and information products without regard to the data's physical location.
Web users are able to place orders through the NVDS system, pay for the
data with a credit card, and receive immediate access to these data,
without manual intervention by customer service representatives. NVDS
currently processes approximately 60,000 orders per year and manages
more than \$2.5 million in user fee transactions. NVDS provides direct
access to some datasets and points to partner data systems for others.
NVDS points to CLASS to provide access to satellite data and points to
NOMADS for access to gridded model data.

The NCDC Web access pages provide access to the NEXRAD Level II and
Level III data through the HDSS Access System, which provides a
functional, if somewhat cryptic, interface. However, at the time of
writing (mid August 2004) NVDS provided no links to NEXRAD data other
than national mosaic reflectivity.

As CLASS expands to provide access to radar and model data, its
interfaces with NVDS and NOMADS must be enhanced and maintained to
ensure all of the links are fully functional and up to date.

NVDS can be accessed via <http://www.nvds.noaa.gov/>.

[Web CliServ]{.underline}

Number range CHAPTEROne of the first steps in accessing in situ climate
data is usually determining the location of a nearby weather station.
This is not a simple task, since more than 30,000 weather stations have
been in operation in the United States and an even greater number
worldwide. Adding to the complexity, over time these weather stations
open and close, change location, and change instrumentation. Web CliServ
was developed in 1997 to provide Web access to NCDC's vast station
history and climate inventory databases.

An example of a common Web CliServ application is the need for a
customer to determine the existence of climate data for a given location
either yesterday or many years ago. Using Web CliServ, the customer
enters the city name, zip code, county, or state to get a list of
weather stations for that geographic area. Once a weather station is
selected, a history of that station's record is presented along with a
list of data types available, inventories, and data documentation. Web
Cliserv is linked to other web access tools and the Online Store to
provide a seamless Web system for NCDC customers.

[NESDIS OSDPD Product Access]{.underline}

The NESDIS Office of Satellite Data Processing and Distribution (OSDPD),
in collaboration with Office of Research and Applications, provides
access to a variety of satellite products, mainly as maps or images, via
an easy to use Web interface. For the most part, only relatively recent
products are accessible through this system. However, there are
exceptions. For example, monthly mean sea surface temperature charts are
available back to 1984. The available products include:

-   Maps of sea surface temperature, degree heating weeks, and ocean
    surface winds

-   Weekly, monthly, and daily contour plots and global analyses of
    aerosol optical thickness

-   Daily and monthly mean global maps of outgoing infrared and absorbed
    and available incoming solar radiation

-   Vegetation Index and SSM/I derived products

-   AVHRR and AMSU composites

-   MODIS and GOES imagery and product images

-   Movie loops that show daily changes in a number of retrieved ATOVS
    data products as well as the accuracy of the various ATOVS systems.

[NCDC Climate Data On-line]{.underline}

During 1998-1999, NCDC developed the NOAA National Data Center Climate
Data Online System to provide fast, easy online access to a variety of
datasets of climate data from in situ instruments. This system provides
access to both recent and historical data, useful for studies of
particular weather events, and for historical analysis of data for
statistical and other research purposes. The types of data currently
included in the system, which continues to be populated, are surface
daily and monthly data, hourly precipitation data, 15-minute
precipitation data, global surface hourly, and global upper air data.

Data are accessible and selectable by region, country, state (US),
county (US), climate division (US), and station, and by time period
(year, month, day, hour). The system presents an intuitive Web interface
using maps and lists. It is quite flexible in that you can select, for
example, all stations in a county, or just pick selected stations from a
particular state. For some datasets, the system allows users to select
just the weather elements (e.g., precipitation, temperature) that they
desire. It also allows users to select from a number of possible formats
for data delivery.

Climate Data Online includes a complete help system with, general system
information, detailed system help for diagnosing problems (e.g.,
inability to connect due to firewall on user end), format documentation,
data samples, station lists, utility software, etc. This system is
connected to NVDS, which allows for a charging mechanism for data
transactions, similar to that which is required for CLASS.

More information is available at
<http://www1.ncdc.noaa.gov/pub/data/special/ams-2000-pap2.pdf>.

[NCDC CLIMVIS]{.underline}

The Climate Visualization System (CLIMVIS) is an interactive graphing
tool designed to allow visual browsing of the data available on-line at
NCDC via a standard Web browser. Using the CLIMVIS system, data can be
viewed as time series graphs or charts for individual sites or as
contoured products for various regions of the United States. Users
select the location, time and parameters wanted, and the charts are
automatically produced and displayed as images.

CLIMVIS can be accessed via
<http://www.ncdc.noaa.gov/oa/climate/onlineprod/drought/xmgr.html>.

[NGDC GEODAS]{.underline}

The Geophysical Data System (GEODAS) is the legacy interactive database
management system developed by the National Geophysical Data Center
(NGDC) for use in the assimilation, storage and retrieval of geophysical
data. On-line versions of GEODAS allow free downloads of geophysical
data. GEODAS Search and Data Retrieval gives access to the Marine
Trackline Geophysics and NOS Hydrographic Surveys databases, allowing
searches and downloads by geographic area, year of survey, institution,
platform, cruise, or geophysical parameter. The GEODAS Grid Translator
allows users to create and download a custom grid of
bathymetry/topography from ETOPO2 Worldwide Gridded Relief, US Coastal
Relief Model and Great Lakes Bathymetry, with user-defined options for
area, format, cell size, etc.

GEODAS can be accessed via
<http://www.ngdc.noaa.gov/mgg/geodas/geodas.html>.

[NGDC Interactive Map Services]{.underline}

NGDC\'s interactive map services provide visual display of one or more
data layers of selected datasets archived in geospatial databases at
NGDC, often with links to download the underlying data. Users can view
layers containing different types of data, adding reference information
such as continents, rivers, and political boundaries if desired. Zoom,
pan, identify, search, find, and hyperlink capabilities are also
available. The maps are implemented through an ESRI ArcIMS Interface.

NGDC Interactive Maps can be accessed via
<http://www.ngdc.noaa.gov/maps/interactivemaps.html>.

[NGDC SPIDR]{.underline}

SPIDR is a service provided by the World Data Centers for
Solar-Terrestrial Physics, which allows users to browse, plot, retrieve
and perform data-mining operations on solar-terrestrial physics data.
SPIDR is a distributed network of synchronous databases in the United
States, Russia, Japan, Australia and South Africa. The key concepts of
the site are the data basket (a collection of different space weather
parameters selected from different databases for the same time interval)
and the space weather event. The data basket allows users to manipulate
and receive the data in various standard formats for easy integration
into existing tools. The "event" system is designed to allow the user to
specify desired spatial, temporal, and parameter conditions in fuzzy
linguistic and/or numeric terms and then to mine the archives and
receive a ranked list of space weather events best matching the desired
conditions in the historical archive.

SPIDR can be accessed via <http://spidr.ngdc.noaa.gov/spidr/index.html>.

[NODC Archive Management System]{.underline}

The NODC Archive Management System (AMS) enables datasets to be
accessioned, archived, and disseminated via Web browsers. The AMS
includes an interface to the NMMR and the NODC Ocean Archive System,
which allows for the search and dissemination of datasets archived at
NODC. Because the Ocean Archive System only searches for original data
sets in the NODC archives, NODC provides a number of other search
utilities to specific products, such as the NODC World Ocean Database
(via World Ocean Database Select and Search System). The current
functionality of the Ocean Archive System allows a user to obtain the
entire directory for an accession. When fully implemented, it will allow
a user to retrieve specific files.

The AMS can be accessed via <http://www.nodc.noaa.gov/Archive/Search/>.

[NCDDC MERMAid]{.underline}

The Metadata Enterprise Resource Management Aid (MERMAid) was developed
by the National Coastal Data Development Center (NCDDC) to provide
coastal data resources (organizations and individuals) with a tool to
develop, validate, manage and publish metadata records via secure
Internet access. MERMAid allows users/data providers to establish
unlimited metadata databases to organize their metadata records any way
they see fit (i.e. by program, project, data type, personnel). Some of
the key features in MERMAid include (1) user-defined roles and
permissions at the metadata management and database levels; (2) change
tracking; and (3) enhanced validation. Existing FGDC compliant metadata
(in XML format) can be ingested into and managed through MERMAid.

In the near future, NCDDC will be shifting from its current metadata
catalog to a knowledge base catalog. To better leverage these new
capabilities, enhanced search and discovery tools will be made available
to the public and metadata managers.

MERMAid can be accessed at <http://www.ncddc.noaa.gov/Metadata/Tools>.

[PMEL Live Access Server]{.underline}

The Live Access Server (LAS) developed by the Pacific Marine Environment
Laboratory is a highly configurable Web server designed to provide
flexible access to geo-referenced scientific data. It can present
distributed data sets as a unified virtual database through the use of
OPeNDAP (see section 3.3).

LAS enables Web users to

-   Visualize data with graphics generated on demand

-   Request custom subsets of variables in a choice of file formats

-   Access metadata pertaining to the data

-   Compare (difference) variables from distributed locations

LAS enables data providers to

-   Unify access to multiple types of data from a single interface

-   Create thematic data servers from distributed data sources

-   Offer derived products on the fly

-   Remedy metadata inadequacies

-   Offer unique products (e.g. visualization styles specialized for the
    data)

Further information on LAS is available from the Pacific Marine
Environmental Laboratory (PMEL) at
(<http://ferret.pmel.noaa.gov/Ferret/LAS/ferret_LAS.html>).

**Appendix 3**

**Relevant Guidelines and Standards**

**Metadata standards**

OMB Circular A-16 (Revised) was issued in August 2002. It provides
direction concerning the Federal Geographic Data Committee (FGDC) and
the National Spatial Data Infrastructure (NSDI). The circular requires
the development, maintenance, and dissemination of a standard core set
of digital spatial information for the Nation. This set consists of
"themes" of national significance. Climate is among the themes
specifically identified. It is defined as "data describing the spatial
and temporal characteristics of the Earth's atmosphere/hydrosphere/land
surface system. These data represent both model-generated and observed
(either in situ or remotely sensed) environmental information." Thus, it
is clear that the data managed by CLASS are considered part of the NSDI.

**[FGDC CSDGM]{.underline}**

To facilitate exchange of data, standards are needed to define the
information that is necessary to describe dataset that are part of the
NSDI. This information, or metadata, should include all information
necessary for prospective users to ascertain the dataset's availability,
its fitness for an intended use, and the means of accessing,
transferring and using it. The FGDC Content Standard for Digital
Geospatial Metadata (CSDGM)
(<http://www.fgdc.gov/metadata/contstan.html>) was developed in the mid
1990s to meet this need. It specifies an extensive list of elements to
define information about a dataset's contents, availability, lineage,
processing history, sources, and intended use, among others. Executive
Order 12906, \"Coordinating Geographic Data Acquisition and Access: The
National Spatial Data Infrastructure,\" was issued in 1994. Section 3,
paragraph (b) states: \"Standardized Documentation of Data. Beginning
nine months from the date of this order, each agency shall document all
new geospatial data it collects or produces, either directly or
indirectly, using the standard under development by the FGDC, and make
that standardized documentation electronically accessible to the
\[National Spatial Data\] Clearinghouse network. Within one year of the
date of this order, agencies shall adopt a schedule, developed in
consultation with the FGDC, for documenting, to the extent practicable,
geospatial data previously collected or produced, either directly or
indirectly, and making that data documentation electronically accessible
to the Clearinghouse network.\"

A set of remote sensing extensions to the FGDC metadata standard was
developed with broad public participation during the late 1990s. They
were approved and adopted in October 2002. These extensions expand the
information already provided by the base standard to better accommodate
remotely sensed data. The extensions, FGDC Content Standard for Remote
Sensing Swath Data, ([http://www.fgdc.gov/standards/status/
csdgm_rs_ex.html](http://www.fgdc.gov/standards/status/csdgm_rs_ex.html))
specify association of data with its date, time, and geolocation. While
geolocation information that varies from measurement to measurement is
included as part of the data, much of the descriptive information does
not change, such as the parameters of a satellite orbit or the
orientation of the instruments and optical systems on the platform. Such
information is more appropriately stored as metadata than as data, and
the metadata necessary to derive the geolocation information essential
to the swath standard are included in the Remote Sensing Extensions.

The requirement defined in Executive Order 12906 is reflected in the
NESDIS Policy on Metadata and Geospatial Data: \"therefore, it is
incumbent upon all program managers within all NESDIS Offices and
Centers, who are responsible for data set products, to have them
properly documented with FGDC-formatted metadata\", and in the NOAA
Administrative Order on Management of Environmental and Geospatial Data
and Information: \"NOAA environmental and geospatial data will be
maintained in accord with applicable Office of Management and Budget
(OMB) regulations, including OMB Circulars A-16 and A-130; Federal
Geographic Data Committee (FGDC) approved data standards\".

**[ISO 19115]{.underline}**

Over the past few years, the ISO has developed an international standard
for metadata, formally known as the International Standard for
Geographic Information -- Metadata (ISO 19115). The American National
Standards Institute (ANSI) adopted this standard as an ANSI standard
without changes in late 2003.

The ISO standard used the FGDC standard as its starting point but
included several modifications resulting from experience with FGDC
CSDGM. The ISO standard:

-   Includes more elements overall and many more "Optional" elements

-   Addresses many known deficiencies in FGDC CSDGM (i.e.
    Raster/Imagery, granularity, free-text)

-   Is formally defined through Unified Modeling Language

-   Supports multi-level metadata with inheritance

-   Supports creation of multi-lingual metadata and searches across
    multiple languages

-   Supports interoperability through structured relationships and more
    reliance on fixed code lists rather than free text.

ISO 19115 includes a formal process for extensions, which may include
new elements or extended domains. It also supports definition of
community profiles, which consist of

-   Core (mandatory) elements

-   Additional elements mandatory within the profile, which are selected
    from the total defined within the standard

-   Possible extensions

The relationship between the standard, a comprehensive profile, and a
community profile is illustrated in Figure A3.

![](media/image3.wmf){width="3.566666666666667in"
height="2.509027777777778in"}

Figure A3. ISO Metadata Profiles

ISO 19115 specifies the structure and content of geospatial metadata and
includes an annex with a sample implementation in XML. However, the XML
implementation provided is not part of the standard. Consequently, the
actual execution of the standard could vary based on its interpretation.
Recognizing that a standard for implementation would contribute to
interoperability, ISO is developing standard 19139 - Geographic Metadata
Implementation Model. This new standard, which is still being developed,
will define a single UML interpretation and unambiguous XML schema for
implementing ISO 19115.

[FGDC CSDGM versus ISO 19115]{.underline}

The next version of the FGDC standard (version 3) will be a form of the
international standard. Formal acceptance of this new version of the
FGDC CSDGM is expected in early 2005. The FGDC has sponsored the
development of the FGDC-ISO Crosswalk Tool, which will convert FGDC
metadata to ISO Metadata format. The tool will be available free of
charge once it is accepted.

In early 2003 ISO confirmed the approval of a new work item for 19115,
Geographic information - Metadata - Part 2: Extensions for imagery and
gridded data. The FGDC-CSDGM Extensions for Remote Sensing Metadata are
the starting point for this work. A new Draft International Standard
incorporating these extensions is scheduled to be finalized by September
2005.

**Spatial and Gridded Data Standards**

**[Federal Information Processing and ANSI Standards]{.underline}**

The National Institute of Standards and Technology (NIST) has issued
Federal Information Processing Standards (FIPS) under the authority of
the Brooks Act (replaced by the Clinger-Cohen Act of 1996). Further to
this is the National Technology Transfer and Advancement Act of 1995 (PL
104-113), which is implemented through OMB Circular Number A-119. OMB
Circular A-119 indicates that agencies should use voluntary consensus
standards, except when that use is impractical. NIST now seeks to issue
FIPS only for those cases where a Federal Government unique requirement
exists, such as in security. NIST has been seeking to withdraw FIPS
where there exists either an equivalent voluntary consensus standard, or
when the standard has become obsolete, or both. As a result, a large
number of FIPS have been withdrawn, although many are still kept on-line
as reference material (<http://www.itl.nist.gov/fipspubs>).

The Spatial Data Transfer Standard was issued as FIPS 173-1 in 1994.
However, since there is now an equivalent ANSI standard, this has been
withdrawn and superseded by ANSI National Committee on Information
Technology Standards (NCITS) 320-1998 Spatial Data Transfer Standard
(SDTS). SDTS defines a general mechanism for the transfer of
geographically referenced spatial data and its supporting metadata. The
overriding principle of SDTS is that the spatial data transfer should be
self-documenting. The data set should contain all of the information
that is needed to assess and/or use the data for any appropriate
GIS-type application.

Most relevant to CLASS is SDTS Part 5: Raster Profile and Extensions,
accepted as an FGDC Standard in February 1999. Significant effort was
dedicated to harmonize SDTS with other standards, particularly the Earth
Observing System Data and Information System (EOSDIS) Satellite Imagery
suite of standards. A key concept implemented in the Raster Profile and
Extensions is the use, where appropriate, of existing image transfer
standards to encode the actual image data, such as Tagged Image File
Format (TIFF) or JFIF (JPEG File Interchange Format). SDTS modules
supply the means to register the image data geographically as well as
supply any other metadata necessary for its use in a geographic
information or analysis setting.

**[ISO Standards]{.underline}**

ISO is undertaking a number of activities concerning gridded or image
data exchange standards that are potentially applicable to CLASS.

-   ISO 12175 Space data and information transfer systems \-- Standard
    formatted data units \-- Structure and construction rule, 1994

-   Technical Report ISO/TR 19121 on imagery and gridded data was
    published in October 2002. It outlines standards currently used for
    imagery and describes further needs for standardization in imagery
    and gridded data.

-   ISO 19129 standard for imagery, gridded and coverage data framework
    is under development.

-   ISO 19130 standard for sensor and data models for imagery and
    gridded data is under development. It will specify a sensor model
    describing the physical and geometrical properties of each kind of
    remote sensor that produce imagery and will define a conceptual data
    model that specifies the minimum content requirement and
    relationships among the components to make it possible to geolocate
    and analyze the data.

Although it has not yet been accepted as an International Standard ISO
19130 could be very important to the entire remote sensing community.
CLASS should carefully monitor its progress and consider its application
once it has been accepted and endorsed by the producer or user
communities.

**[WMO Standards]{.underline}**

The World Meteorological Organization (WMO) specifies a number of
standard formats and data representation forms for exchange of data and
products within and between the meteorological and hydrological
communities. Use of WMO standards is not mandated within Federal
Agencies. However, WMO standards for exchange of gridded data products
and imagery are widely used.

GRIB (GRIdded Binary) was primarily designed by WMO to exchange gridded
data generated by numerical weather prediction models. It is an
efficient method for transmitting and archiving large volumes of
two-dimensional meteorological and oceanographic data and is widely used
for storage and exchange of gridded data within the meteorological
community. It is the standard used by all of the operational
meteorological centers (e.g. NCEP and the European Center for
Medium-Range Weather Forecasts) for exchange of their gridded data and
forecasts. A newer version of GRIB, called GRIB2, is being used by
NOAA\'s Weather Service to distribute the gridded National Digital
Forecast Database.

BUFR (Binary Universal Form for the Representation of Meteorological
Data) is a binary format designed for the exchange of meteorological
point data. BUFR files are stream-based and consist of a number of
consecutive records. A BUFR record containing observational data of any
sort also contains a coded description of what those data are: the
description includes identifying the parameter in question, (height,
temperature, pressure, latitude, date and time, whatever), the units,
any decimal scaling that may have been employed to change the precision
from that of the original units. Like GRIB, this description is encoded
in tables which comprise the major part of the BUFR documentation, with
the consequence that BUFR data cannot be read without application of the
appropriate tables. BUFR is widely used for the exchange of
observational data in the operational meteorological and oceanographic
communities.

**[CEOS Standards]{.underline}**

*[CEOS format]{.underline}*

The aim of the Committee on Earth Observation Satellites (CEOS)
superstructure format, usually referred to as simply CEOS format, is to
minimize the effort to read and write data products from similar Earth
observation sensors. This is achieved by establishing a standard for a
family of formats, and then making further recommendations for specific
sensor classes (i.e. optical and SAR sensors).

CEOS has approved and recommended many implementations of the
superstructure, and has also developed a CEOS SAR format, as a specific
implementation of the CEOS superstructure concept. CEOS has also
coordinated implementations of the CEOS Superstructure for NOAA AVHRR
data, passive microwave, scatterometer and altimeter data.

The CEOS superstructure is a general-purpose concept that can be applied
to any type of data. It can be used to label and describe data,
metadata, and the format of the data. CEOS uses a variety of types of
files to describe different types of data. In the CEOS superstructure,
label information is described in the volume directory file. Text
records can also be used to describe any information on data, and are
usually located within the volume directory file. Metadata such as
sensor and processing information and radiometric and geometric
correction procedures are described in a leader file. Metadata such as
calibration, quality summary, histograms, and detailed processing
summary are described in a trailer file. However, the handling of data
description information is regarded as poor in CEOS format. When data is
interchanged in CEOS format, there is a strong reliance on paper
documentation to describe the parameters, especially in the Leader and
Header files.

*[CEOS Baseband Data Archive Interchange Format]{.underline}*

The CEOS Working Group on Information Systems and Services is currently
exploring development of a Baseband Data Archive Interchange Format
(CEOS ICF). This is driven by a need to exchange data between different
archiving facilities, which do not employ the same processing systems.
CEOS ICF would be an intermediate format that all archive centers could
interpret. Facilities would translate their own format into CEOS ICF to
export their archive data and would translate CEOS ICF into their own
format in order to import data from other facilities.

**[Industry and de-facto standards]{.underline}**

*[NetCDF]{.underline}*

NetCDF (network Common Data Form) is an interface for array-oriented
data access and a freely distributed collection of software libraries,
defining an application programmer interface (API). The netCDF libraries
also define a machine-independent format for representing scientific
data. Together, the interface, libraries, and format support the
creation, access, and sharing of scientific data. NetCDF was developed
at the Unidata Program Center in Boulder, Colorado.

Many groups within the scientific community have adopted netCDF as a
standard way to represent scientific data. Since the software is
available via anonymous FTP Unidata states that it is difficult to know
where and how netCDF is being used. However, it notes that over 2000
distinct hosts in 55 countries have downloaded the netCDF software
distribution since May 1997. More than 60 software packages are
available for manipulating or displaying netCDF data. These include such
well-known packages as EPIC, FERRET, GrADS and IDL.

Further information on netCDF is available from Unidata at
<http://my.unidata.ucar.edu/content/software/netcdf/index.html>.

*[HDF and HDF5]{.underline}*

HDF (Hierarchical Data Format) software and file formats have been
developed for facilitating access to scientific data by the National
Center for Supercomputing Applications (NCSA). The HDF software includes
I/O libraries and tools for analyzing, visualizing, and converting
scientific data and can be used for storing images, multi-dimensional
arrays, etc. The software, developed and supported by NCSA, is available
free of charge.

HDF is used world wide in many fields, including Environmental Science
and Aerospace. Most importantly for CLASS, NASA selected HDF as the
basis for developing the standard format for \\EOSDIS.

It must be noted that there are two HDF formats, HDF (4.x and previous
releases) and HDF5. Although both of these formats are supported by
NCSA, HDF and HDF5 are completely different and are [not]{.underline}
compatible. HDF5 was developed to resolve shortcomings of HDF that
became apparent as demand grew for bigger and faster data storage
systems, handling larger and more complex data structures. HDF5 is more
scalable, with an improved data model and a more flexible robust
software library with increased emphasis on remote and distributed
access to data.

Both HDF and HDF5 support data "chunking" and compression by chunks. If
implemented in accordance with appropriate user requirements, this
enables dramatically faster selection of subsets of the data. By
subdividing the large arrays and individually compressing each piece,
small regions can be individually selected and decompressed. Without
this feature, an entire large array would need to be retrieved and
decompressed even if only a small piece is wanted.

Information on HDF and HDF5 is available from NCSA at
<http://hdf.ncsa.uiuc.edu/>.

*[HDF-EOS]{.underline}*

Although HDF meets many NASA specifications for accessing data, EOS
applications required additional conventions and data types, which led
to the development of HDF-EOS. HDF-EOS employs standard HDF objects,
including images, tables, text, and data arrays but it also defines
three additional data types based on HDF objects: grid, point, and
swath. These data types allow the file contents to be referenced to
Earth coordinates, such as latitude and longitude, and to time. Grid
data types place the data on grids using one of many standard
projections. Swath data types represent data that is ordered in time.
Point data structures represent data that are irregularly spaced in time
and geolocation, such as weather station data or instrument measurements
taken on buoys.

Basically, HDF-EOS data sets are HDF objects with the added feature of
being able to support geolocation information. This geolocation
information is included in the HDF files through the automatic inclusion
of structural metadata. One other feature that specifically defines an
HDF-EOS data set is the mandatory inclusion of EOSDIS Core System core
metadata. This is not required in ordinary HDF files. It should also be
noted that an HDF file may include both HDF and HDF-EOS data objects and
that HDF-EOS data sets can be read by HDF tools and routines but without
access to the geolocation information.

HDF-EOS is currently being used for data being gathered by NASA\'s EOS
satellites. Terra and called Aqua utilize HDF-EOS 4, while Aura relies
on HDF-EOS 5.

Additional information on HDF-EOS 4 and 5 is available from NASA at
<http://hdfeos.gsfc.nasa.gov/hdfeos/index.cfm>.

*[Native formats]{.underline}*

In some cases, it is necessary to provide data in the native format in
which the producer provides them. In those cases, users should be given
the option of downloading decoder software to accompany the data.

*[Radar data formats]{.underline}*

Several formats are used specifically for exchange of radar data in
radar coordinates. Universal Format (UF) was developed at NCAR in the
late 1970s and is still commonly used within the scientific community.
Several software tools and converters are available for UF.

The overwhelming volume of data from the NEXRAD WSR-88D network has made
its native formats for level II and level III data de facto standards in
the USA. As such, software to read these formats is widely available.
However, the first step that most users take after receiving these data
is to convert it to another format such as UF or NetCDF.

The WMO has developed a standard template for exchange of radar data in
BUFR (Binary Universal Form of Representation of meteorological data).
While this provides an extremely compact and efficient mechanism for
exchange of radar data, it is currently not being used outside of
Europe.

*[Image formats]{.underline}*

GRIB, netCDF, HDF and native formats can provide users with access to
the actual data contained within a multi-dimensional grid. This is often
what scientific and technical users require. However, sometimes it is
best if images or maps are delivered in an image or video format, which
is suitable for display but not quantitative analysis.

The growth of the World Wide Web and its associated software (e.g.
browsers) has led to a number of formats for imagery and video becoming
de-facto standards. JPEG (Joint Photographic Experts Group), PNG
(Portable Network Graphics) GIF (Graphics Interchange Format) and TIFF
are the most commonly used for images. For photographs and full color
imagery, JPEG is used most often. Although JPEG\'s lossy compression can
introduce visible artifacts, these can be minimized, and the savings in
file size is much better than is generally possible with lossless
formats. However, JPEG does not offer support for palette-based images.
But the JPEG 2000 specification does support this capability.

GIF has two capabilities that are particularly important, transparency
and animation. GIF allows areas of images to be tagged as transparent,
allowing multiple images to be superimposed. Also, multiple images
comprising scenes of an animation can be included within a single GIF
file and most browsers include support to display these animations,
although GIF animations are not efficient for medium to large images, or
for animations with many frames.

Geographic TIFF (GeoTIFF) is worth noting as a relatively new standard.
GeoTIFF is mainly a metadata extension of the TIFF format that defines
an interchange format for georeferenced raster imagery. It supports an
extensive list of map projections. The GeoTIFF specification defines a
set of TIFF tags that describe all cartographic information associated
with TIFF imagery that originates from satellite imaging systems,
scanned aerial photography, scanned maps, or digital elevation models.
Its aim is to pride a mechanism to tie a raster image to a known model
space or map projection, and to describe the projection used. It is
important to note that the GeoTIFF specification supports rasters made
up of floating point values, so it can be used for actual data as well
as images.

The GeoTIFF format is completely open and non-proprietary. There is no
restriction on licensing, implementation, promulgation, or any use of
the format. The specifications for GeoTIFF are public; there are
abundant free software source libraries, toolkits and data samples.
Technical support is available through an email forum. Further
information on GeoTIFF is available at
<http://www.remotesensing.org/geotiff/geotiff.html>.

Several formats are used for exchange of video information over the Web.
MPEG (Moving Picture Experts Group), AVI (Audio Video Interlaced), and
MOV (Apple Quicktime) are the most common, with variations of MPEG being
by far the most popular.

**Network API and Protocol Standards**

An Application Programmer Interface (API) allows programmers to work
with data files without having to know the details of the file
structure, organization or content. In this way it greatly simplifies
access to the data and isolates the programs from the files so that
programs do not need to be modified even if the file or content change.
Over the past few years, some APIs have been extended to include network
access, further abstracting the interface to the data and thus hiding
the file location as well as its structure from the application program.

**[OPeNDAP]{.underline}**

OPeNDAP (the Open source Project for a Network Data Access Protocol) is
a non-profit corporation established to develop and promote software
that facilitates access to data via the Internet. OPeNDAP software is
freely available.

OPeNDAP provides a way for researchers to access data anywhere on the
Internet from a wide variety of new and existing programs. By developing
network versions of commonly used data access API libraries, such as
netCDF, HDF and others, OPeNDAP can capitalize on years of development
of data analysis and display packages that use those APIs, allowing
users to continue to use programs with which they are already familiar.

The OPeNDAP architecture uses a client/server model, with a client that
sends requests for data out onto the network to some server, which
answers with the requested data. This is the same model used by the Web
where browsers submit requests to servers for the data that make up Web
pages. OPeNDAP clients can do much more than render and display this
data. Using flexible data types suitable for many uses, including
scientific data, the OPeNDAP servers deliver real data directly to the
client program in the format needed by that client.

To expand the range of data available to a user, OPeNDAP incorporates
data sub-sampling and translation facilities, so that data may be stored
in data structures and formats defined by the data provider, but may be
accessed by the user in a manner identical to the access of local data
files on the user\'s own system. Though there are limitations on the
types of data that may be translated, the facility is flexible and
general enough to handle many of the possible translations.

Currently, OPeNDAP uses Remote Procedure Calls for its communication but
support for SOAP (see below) is being actively considered.

Further information on OPeNDAP is available at
<http://www.opendap.org/>.

**[SOAP]{.underline}**

Simple Object Access Protocol (SOAP) is a simple Extensible Markup
Language (XML) based protocol to support exchange of information between
applications via Hypertext Transport Protocol (HTTP). Essentially, it is
a protocol for accessing a Web Service. SOAP is fundamentally a
stateless, one-way message exchange paradigm, but applications can
create more complex interaction patterns (e.g., request/response,
request/multiple responses, etc.) by combining such one-way exchanges
with features provided by an underlying protocol and/or
application-specific information. SOAP provides the framework by which
application-specific information may be conveyed in an extensible
manner.

Until SOAP was developed in 2001, applications communicated over TCP/IP
networks using Remote Procedure Calls between objects. However, Remote
Procedure Calls represent a compatibility and security problem.
Firewalls and proxy servers will normally block this traffic. Since HTTP
is supported by all Internet browsers, servers and firewalls it provides
a better mechanism to communicate between applications. SOAP has been
developed to accomplish this. It allows application to application
communication via HTTP using messages in well-defined XML. SOAP provides
a way to communicate between applications running on different operating
systems, with different technologies and programming languages.

Further information on SOAP can be found at
<http://www.w3.org/2000/xp/Group/>.

**[WSDL]{.underline}**

Web Services Description Language (WSDL) makes it easier to realize the
benefits of SOAP by providing a way for Web service providers and users
of such services to work together. SOAP is a standard for communicating
content, but without a standard language for describing that content. As
communications protocols and message formats are standardized in the Web
community, it becomes increasingly possible and important to be able to
describe the communications in some structured way. WSDL addresses this
need by defining an XML grammar for describing network services.

Further information on WSDL is available at <http://www.w3.org/TR/wsdl>.

**[Z39.50]{.underline}**

Z39.50 refers to the International Standard, ISO 23950: \"Information
Retrieval (Z39.50): Application Service Definition and Protocol
Specification\", and to ANSI/NISO Z39.50. The standard specifies a
client/server-based protocol for searching and retrieving catalog
information from remote databases. It is interoperable with a broad
range of information resources and services, including libraries and
information services worldwide as well as the Clearinghouse catalogues
supported across the Global Spatial Data Infrastructure.

Further information on Z39.50 is available at
<http://www.loc.gov/z3950/agency/>.

**[CORBA]{.underline}**

Common Object Request Broker Architecture (CORBA) is a
vendor-independent architecture and infrastructure that computer
applications can use to work together over networks. A CORBA-based
program from any vendor, on almost any computer, operating system,
programming language, and network, can interoperate with a CORBA-based
program on almost any other computer, operating system, programming
language, and network.

The separation of interface from implementation is the essence of CORBA.
The interface definition is independent of programming language, but
maps to all of the popular programming languages such as C, C++, Java,
COBOL, and Python. Clients access objects only through their advertised
interface, invoking only those operations that the object exposes
through its CORBA-defined interface, with only those parameters (input
and output) that are included in the invocation.

The CORBA specification was developed by the Object Management Group, an
industry group representing computer manufacturers, independent software
vendors and a variety of government and academic organizations. Thus,
CORBA specifies an industry/consortium standard, not a formal ANSI/ISO
standard.

For more information on CORBA, see
<http://www.omg.org/gettingstarted/corbafaq.htm>.

**[OGC Web Map Server]{.underline}**

The Open GIS Consortium, Inc. (OGC) is a non-profit, international
consortium of more than 250 companies, government agencies and
universities participating in a consensus process to develop publicly
available specifications for exchange and processing of geospatial data.
Open interfaces and protocols defined by OGC specifications allow
developers to "geo-enable" Web and mainstream information services. OGC
Implementation Specifications, available over the Internet, detail the
agreed upon interfaces that OGC develops through its consensus process.
These are software engineering specifications that allow software
developers to build products that implement one or more of the OGC
specifications. Many OGC specifications are pertinent to CLASS: Web Map
Servers, Feature Servers, Coverage Severs and Catalog Services.

The OGC Web Map Server protocol defines a simple HTTP interface for Web
based mapping applications. The Web Map Server provides protocols to
support requests for, and descriptions of, server capabilities and
layers as well as the creation and display of registered and
superimposed map-like views of information. An OGC \"map\" is defined as
a visual representation of data; not the data itself. These maps can be
delivered in a pictorial format such as PNG, GIF or JPEG, or even as
vector-based graphical elements in Scalable Vector Graphics or Web
Computer Graphics Metafile formats.

The protocol is based on simple query syntax for posting a request for
the desired layers and window to the server, which returns a map as a
standard picture. When two or more maps are produced with the same
bounding box, spatial reference system, and output size, the results can
be accurately layered to produce a composite map. Uses of vector
graphics or transparent areas in GIF images allow lower layers to be
visible even when multiple images are superimposed. This provides the
powerful capability for a user to render maps with layers that come
simultaneously from multiple remote and heterogeneous sources. For
example, a map of political boundaries could come from one source,
surface topography from another, and forecast storm-surge height from a
third.

The Web Feature and Coverage specifications go beyond the Map Server
Specification in that they support transmission of actual data rather
than just pictures of data. The feature specification includes points,
lines, and polygons with associated attributes. The Coverage
Specification includes rasters. These specifications prescribe the
formats of the returned data with OGC Geographic Markup Language (GML)
being used for features and several image formats being used for
coverages (geoTIFF is the most popular).

Further information and details are available from the OGC. The OGC Web
Map Server specification is available at
<http://www.opengis.org/docs/01-068r2.pdf> and the Web Coverage Service
specification is available at
http://www.opengeospatial.org/docs/03-065r6.pdf.

The defines the interfaces, bindings, and a framework for defining
application profiles required to publish and access digital catalogues
of metadata for geospatial data, services, and related resource
information. Catalogue services support the ability to publish and
search collections of metadata for data, services, and related
information objects. Metadata in catalogues represent resource
characteristics that can be queried and presented for evaluation and
further processing by both humans and software. Catalogue services
support the use of one of several identified query languages to find and
return results using well-known content models (metadata schemas) and
encodings.

This specification defines a common query language (based on SQL) for
sending requests to catalog systems. This includes bindings with Z39.50,
CORBA and HTTP protocols.

Further information and details of the is available from the OGC at
<http://www.opengis.org/specs/?page=specs>.

**[ESML]{.underline}**

Earth Science Markup Language (ESML) is a "middleware" layer with an
accompanying API developed to address the problem of data/application
interoperability that has long existed in the science community. ESML is
a description of the structure and content of a data file, formalized so
that an application can use it as a guide to automatically read and
decode the file. Thus, using ESML and its API, applications can
understand and use a data file regardless of its format. Since ESML
descriptions are external files, both data producers and data consumers
can write them. A producer would write an ESML file describing its data
set and a user would write a single reader or data decoder for their
analysis or visualization application utilizing the ESML Library. This
reader should then be able to read and understand any data described by
an accompanying ESML file.

As of mid 2004 a description of ESML has been published and a number of
tools have been developed, including software bindings for several
formats including HDF5, HDF-EOS, netCDF, GRIB, ASCII, and NEXRAD.

**Note:** ESML is [not]{.underline} an approved standard and has not yet
been widely adopted.

Further information on ESML is available at
<http://esml.itsc.uah.edu/index2.html>.

**Appendix 4**

**Applicable Documents and References**

Barkstrom, Bruce R., *A Procedure for Developing Engineering
Requirements*, NASA Langley, 5 May 2004

Climate Change Science Program, *Strategic Plan for the U.S. Climate
Change Science Program*, 24 July 2003

Computer Sciences Corporation, *CLASS Archive and Access Requirements*,
20 July 2001

Consultative Committee for Space Data Systems (CCSDS) *Producer-Archive
Interface Methodology Abstract Standard* (CCSDS 651.0-B-1), May 2004

Consultative Committee for Space Data Systems (CCSDS) *Reference Model
for an Open Archive Information System* (CCSDS 650.0-B-1), May 2004

Federal Geographic Data Committee - *Content Standard for Digital
Geospatial Metadata, CSDGM Version 2* (FGDC-STD-001-1998), June 1998

Federal Geographic Data Committee -*Content Standard for Digital
Geospatial Metadata: Extensions for Remote Sensing Metadata*
(FGDC-STD-012-2002), 9 October 2002

GEO *Architecture Subgroup Report* (GEO4DOC 4.1(2)) -- REV 2.0 updated,
13 May 2004

T. Habermann and M.Ohrenschall, *CLASS Gridded Satellite Product
Formats*, October 10, 2002

International Organization for Standardization, *Standard 19115,
Geographic Information -- Metadata*, 2003

NCAR, *Global Change Science Requirements for Long-term Archiving*,
Report of the Workshop, October 28-301998, March 1999

NOAA, *CLASS Allocated Requirements* (CLASS-1017-CLS-REQ-AADS), Version
1.1, 12 November 2003

NOAA, *CLASS Five Year Plan_2003* (CLASS-1013-CLS-PLN-5YEAR), September
2003

NOAA, *CLASS Guidelines for Submission of Data to the Archive as part of
the Archive, Access and Distribution Segment*
(CLASS-1044-CLS-GDL-GUIDE), December 09, 2003

NOAA, *CLASS Master Project Management Plan*, (CLASS-1028-CLS-PLN-MPMP),
Version 1.2, 12 August 2003

NOAA, *CLASS System Requirements* (CLASS-1005-CLS-REQ-SRDOC), Version
1.0, 13 February 2004

NOAA, *NOAA Strategic Plan for FY 2003 -- FY 2008 and Beyond*, March
2003

NRC, Committee on Environmental Satellite Data Utilization, *Utilization
of Operational Environmental Satellite Data: Ensuring Readiness for 2010
and Beyond*, prepublication copy, August 2004

Ocean.US, *Data Management and Communications Plan for Research and
Operational Integrated Ocean Observing Systems*, 10 May 2004

Office of *Management* and Budget, *OMB Circular No. A-16 Revised*, 19
August 2002.

**Appendix 5**

**OAIS Reference Model**

The organization and operation of CLASS should follow the OAIS Reference
Model.

The complete copy of the latest version of the OAIS Reference Model is
available from the CCSDS at
<http://www.ccsds.org/documents/650x0b1.pdf>. The following information
has been extracted from the CCSDS document.

[Terminology]{.underline}

There are several terms in the OAIS Reference Model that have
well-defined meanings within the model. The definitions of these terms
as given in the OAIS Reference Model are reproduced here and are listed
in bold type when they are first referenced in the text.

**Content Data Object**: The Data Object that, together with associated
Representation Information, is the original target of preservation.

**Content Information**: The set of information that is the original
target of preservation. It is an Information Object comprised of its
Content Data Object and its Representation Information. An example of
Content Information could be a single table of numbers representing, and
understandable as, temperatures, but excluding the documentation that
would explain its history and origin, how it relates to other
observations, etc.

**Context Information**: The information that documents the
relationships of the Content Information to its environment. This
includes why the Content Information was created and how it relates to
other Content Information objects.

**Data:** A reinterpretable representation of information in a
formalized manner suitable for communication, interpretation, or
processing. Examples of data include a sequence of bits, a table of
numbers, or the characters on a page.

**Data Management**: The OAIS entity that contains the services and
functions for populating, maintaining, and accessing a wide variety of
information. Some examples of this information are catalogs and
inventories on what may be retrieved from archives, processing
algorithms that may be run on retrieved data, consumer access
statistics, consumer billing, security controls, and OAIS schedules,
policies, and procedures.

**Data Object**: Either a Physical Object or a Digital Object.

**Designated Community**: An identified group of potential Consumers who
should be able to understand a particular set of information. The
Designated Community may be composed of multiple user communities.

**Descriptive Information**: The set of information, consisting
primarily of Package Descriptions, which is provided to Data Management
to support the finding, ordering, and retrieving of OAIS information
holdings by consumers.

**Independently Understandable**: A characteristic of information that
has sufficient documentation to allow the information to be understood
and used by the Designated Community without having to resort to special
resources not widely available, including named individuals.

**Information Object**: A Data Object together with its Representation
Information.

**Information Package**: The Content Information and associated
Preservation Description Information that is needed to aid in the
preservation of the Content Information. The Information Package has
associated Packaging Information used to delimit and identify the
Content Information and Preservation Description Information.

**Open Archival Information System**: An archive consists of an
organization of people and systems that have accepted the responsibility
to preserve information and make it available for a Designated
Community. It meets a set of responsibilities that allows an OAIS
archive to be distinguished from other uses of the term archive. The
term "Open" in OAIS is used to imply that this recommendation and future
related recommendations and standards are developed in open forums, and
it does not imply that access to the archive is unrestricted.

**Packaging Information**: The information that is used to bind and
identify the components of an Information Package. For example, it may
be the ISO 9660 volume and directory information used on a CD-ROM to
provide the content of several files containing Content Information and
Preservation Description Information.

**Preservation Description Information**: The information which is
necessary for adequate preservation of the Content Information and which
can be categorized as Provenance, Reference, Fixity, and Context
information.

**Provenance Information**: The information that documents the history
of the Content Information. This information tells the origin or source
of the Content Information, any changes that may have taken place since
it was originated, and who has had custody of it since it was
originated. Examples of Provenance Information are the principal
investigator who recorded the data, and the information concerning its
storage, handling, and migration.

**Representation Information**: The information that maps a Data Object
into more meaningful concepts. An example is the ASCII definition that
describes how a sequence of bits (i.e., a Data Object) is mapped into a
symbol.

[Reference Model]{.underline}

A clear definition of information is central to the ability of an OAIS
to preserve it. There are many types of information involved in the
long-term preservation of information in an OAIS. Each of these types
can be viewed as a complete **Information Object** in that it contains a
data object and adequate information to understand the data. The sum of
what is needed to interpret the data is collectively known as
**Representation Information** in the OAIS model. Thus, the Information
Object consists of data with its accompanying Representation
Information.

The types of supporting information needed to enable long-term
preservation can be categorized by their content and function in the
operation of an OAIS into **Content Information** objects,
**Preservation Description Information** objects, **Packaging
Information** objects, and **Descriptive Information** objects. Taxonomy
of these components is illustrated the figure and each is briefly
described below.

![](media/image4.wmf){width="5.709722222222222in"
height="2.3694444444444445in"}

OAIS Information Object Taxonomy

The Content Information is the set of information that is the original
target of preservation by the OAIS. It comprises the data together with
its Representation Information.

Archival information must include all of the information necessary to
allow the understanding of the Content Information over an indefinite
period of time. The specific set of information required for this
function, is collectively called Preservation Description Information
(PDI). The PDI must include information that is necessary to adequately
preserve the particular Content Information with which it is associated.
It should be noted that for archival, all classes of PDI information
must be present. This requirement and details of these components and
their relationships is further explained in section 4.2.1.4 of the OAIS
Reference Model.

The Packaging Information is that information which, either actually or
logically, binds or relates the components of the package into an
identifiable entity on specific media. For example, if the Content
Information and PDI are identified as being the content of specific
files on a CD-ROM, then the Packaging Information may include the ISO
9660 volume/file structure on the CD-ROM.

In addition to preserving information, the OAIS must provide adequate
features to allow users to locate information of potential interest,
analyze that information, and order desired information. This is
accomplished through Descriptive Information, which can be viewed as an
index to enable efficient access to the associated information (e.g.
documents or applications that can be used to locate, analyze, retrieve,
or order information from the OAIS).

.

**Appendix 6**

CLASS Dataset Responsibilities

  -------------- ------------- ------------------------------------- --------- --------- ------------------------------------------------------------------------- ----------------------------------------------------------------------- -----------
  **Observing                  **Current or planned Archive/access   **Level   **Level   **Level 2**                                                               **Level 3**                                                             **Level 4**
  System**                     System**                              0**       1**                                                                                                                                                         

  POES           AVHRR         CLASS                                                                                                                                                                                                       

                 HIRS          CLASS                                                                                                                                                                                                       

                 AMSU          CLASS                                                                                                                                                                                                       

                 MHS           CLASS                                                                                                                                                                                                       

                 SBUV          CLASS                                                                                                                                                                                                       

  GOES           Imager        CLASS                                                                                                                                                                                                       

                 Sounder       CLASS                                                                                                                                                                                                       

                 SXI           NGDC                                                                                                                                                                                                        

                 SEM           NGDC                                                                                                                                                                                                        

  DMSP           OLS           NGDC                                                                                                                                TBD                                                                     

                 SSIES         NGDC                                                                                                                                TBD                                                                     

                 SSJ4          NGDC                                                                                                                                TBD                                                                     

                 SSM           NGDC                                                                                                                                TBD                                                                     

                 SSM/I         NGDC & NCDC                                               [5 Standard                                                               TBD                                                                     
                                                                                         products](http://lwf.ncdc.noaa.gov/oa/satellite/ssmi/ssmiproducts.html)                                                                           

                 SSM/T-1       NGDC & NCDC                                                                                                                         TBD                                                                     

                 SSM/T-2       NGDC & NCDC                                                                                                                         TBD                                                                     

                 SSMIS         NGDC                                                                                                                                                                                                        

                 SSULI         Naval Research Laboratory                                                                                                                                                                                   

                 SSUSI         [Johns Hopkins University, Applied                                                                                                                                                                          
                               Physics                                                                                                                                                                                                     
                               Laboratory](http://www.jhuapl.edu/)                                                                                                                                                                         

  Terra          ASTER         NASA EDG                                                                                                                                                                                                    

                 CERES         NASA EDG                                                  [Many standard                                                            [Many standard                                                          
                                                                                         products](http://eosweb.larc.nasa.gov/project/ceres/table_ceres.html)     products](http://eosweb.larc.nasa.gov/project/ceres/table_ceres.html)   

                 MISR          NASA EDG                                                  [Several standard                                                         [Global maps](http://www-misr.jpl.nasa.gov/mission/data.html)           
                                                                                         products](http://www-misr.jpl.nasa.gov/mission/data.html)                                                                                         

                 MODIS         NASA EDG                                                  [30 standard products](http://modis.gsfc.nasa.gov/data/dataproducts.html) [3 standard                                                             
                                                                                                                                                                   products](http://modis.gsfc.nasa.gov/data/dataproducts.html)            

                 MOPITT        NASA EDG & NCAR                                                                                                                                                                                             

  Aqua           AIRS          NASA EDG                                                  [Many standard products](http://aqua.nasa.gov/AIRS/airs_science_dpt.html)                                                                         

                 AMSU          NASA EDG                                                                                                                                                                                                    

                 AMSR-E        NSIDC                                                                                                                               Provisional only                                                        

                 CERES         NASA EDG                                                  [Many standard                                                            [Many standard                                                          
                                                                                         products](http://eosweb.larc.nasa.gov/project/ceres/table_ceres.html)     products](http://eosweb.larc.nasa.gov/project/ceres/table_ceres.html)   

                 HSB           NASA EDG                                                                                                                                                                                                    

                 MODIS         NASA EDG                                                  [30 standard products](http://modis.gsfc.nasa.gov/data/dataproducts.html) [3 standard                                                             
                                                                                                                                                                   products](http://modis.gsfc.nasa.gov/data/dataproducts.html)            

  NPP            ATMS          CLASS                                                                                                                                                                                                       

                 CrIS          CLASS                                                                                                                                                                                                       

                 VIIRS         CLASS                                                                                                                                                                                                       

                 OMPS          CLASS                                                                                                                                                                                                       

  NPOESS         APS           CLASS                                                                                                                                                                                                       

                 CMIS          CLASS                                                                                                                                                                                                       

                 CrIS/ATMS     CLASS                                                                                                                                                                                                       

                 ERBS          CLASS                                                                                                                                                                                                       

                 GPSOS         CLASS                                                                                                                                                                                                       

                 OMPS          CLASS                                                                                                                                                                                                       

                 Radar         CLASS                                                                                                                                                                                                       
                 Altimeter                                                                                                                                                                                                                 

                 SESS          CLASS                                                                                                                                                                                                       

                 TSIS          CLASS                                                                                                                                                                                                       

                 VIIRS         CLASS                                                                                                                                                                                                       

                 Multiple      CLASS                                                     NDE                                                                                                                                               
                 Instruments                                                                                                                                                                                                               

  METOP          AVHRR         EUMETSAT & CLASS                                                                                                                                                                                            

                 HIRS          EUMETSAT & CLASS                                                                                                                                                                                            

                 AMSU          EUMETSAT & CLASS                                                                                                                                                                                            

                 MHS           EUMETSAT & CLASS                                                                                                                                                                                            

                 IASI          EUMETSAT & CLASS                                                                                                                                                                                            

                 GRAS          EUMETSAT & CLASS                                                                                                                                                                                            

                 ASCAT         EUMETSAT & CLASS                                                                                                                                                                                            

                 GOME          EUMETSAT & CLASS                                                                                                                                                                                            

  Radarsat       SAR           CLASS                                                                                                                                                                                                       

  Jason-2/OSTM   POSEIDON-3    TBD                                                                                                                                                                                                         
                 altimeter                                                                                                                                                                                                                 

                 WSOA          TBD                                                                                                                                                                                                         

  NEXRAD                       NCDC HAS                                                                                                                                                                                                    

  NCEP Model                   NCDC NOMADS                                                                                                                                                                                                 
  analyses                                                                                                                                                                                                                                 

  Surface data   Daily         NCDC CDO                                                                                                                                                                                                    

                 Hourly        NCDC CDO                                                                                                                            Included in NCEP model analyses                                         Included in
                                                                                                                                                                                                                                           NCEP
                                                                                                                                                                                                                                           forecasts

  Global                       NCDC CDO                                                                                                                                                                                                    
  Radiosonde                                                                                                                                                                                                                               
  Data                                                                                                                                                                                                                                     

  Marine data    Buoy          NCDC CDO                                                                                                                                                                                                    

                 Ships         NCDC CDO                                                                                                                                                                                                    

                 ICOADS        NCDC CDO                                                                                                                                                                                                    
  -------------- ------------- ------------------------------------- --------- --------- ------------------------------------------------------------------------- ----------------------------------------------------------------------- -----------

+-------+----------------------------+-------+------------------------+
|   --  | Not applicable or not      |   --  | Is not archived        |
|       | produced                   |       |                        |
|   --  |                            |   --  |                        |
+-------+----------------------------+-------+------------------------+
|   --  | Is managed by CLASS        |   --  | Will be managed by     |
|       |                            |       | CLASS                  |
|   --  |                            |   --  |                        |
+-------+----------------------------+-------+------------------------+
|   --  | Will not be managed by     |   --  | Unknown / To be        |
|       | CLASS                      |       | determined             |
|   --  |                            |   --  |                        |
+-------+----------------------------+-------+------------------------+