# Presentation: 310680

## Integrating the ACE and SAMPEX Data Centers into the Emerging Virtual Observatory System:

- Practical experience and perspectives
- Andrew Davis – Caltech
- SAW Workshop, April 2007

## Slide 2

## Data Sets Hosted by

- The ACE Science Center (ASC)
- at Caltech

- Advanced Composition Explorer (ACE): Verified Level 2, Level 3, and unverified Browse data from all 9 instruments.
- SAMPEX: Data are served from the archive using the same tools and services developed for ACE.
- STEREO: In-situ Solar Energetic Particle data (instrument team site).
- ~1 Tbyte total per mission, mostly time-series

## Slide 4

## Becomes Resident Archive after ingest complete

## Data Provider Issues

- None
- The End

## Data Provider Issues

- For ACE, the data pipeline is well-established
- For SAMPEX, the science team has needed to reprocess all the data to fix various errors they have lived with for years...
- All time-series data received from the science teams is converted by the archive to a common format for dissemination and archival purposes
- So this talk is about issues related to        The archive acting as a data provider
- To VOs
- To entities like CDAWeb, CoSEC, etc.

## Archive  VO Issues: Metadata

- Need to repackage ACE & SAMPEX metadata using SPASE data model
- Began working with SPASE model several years ago
- Provided feedback to SPASE development process to ensure the model was rich enough to adequately  describe our data

## SPASE Evolution

- <PhysicalParameter>
- <Name>E2</Name>
- <Description>LEFS150 0.062-0.102 MeV Electron Flux</Description>
- <Units>1/(cm^2 s sr MeV)</Units>
- <Measured>
- <Particle>
- <ParticleType>Electron</ParticleType>
- <ParticleQuantity>No Flux Choice</ParticleQuantity>
- </Particle>
- </Measured>
- </PhysicalParameter>				SPASE 1.0

## SPASE Evolution

- <ParameterKey>E2</ParameterKey>
- <Description>LEFS150 0.062-0.102 MeV Electron Flux</Description>
- <Units>1/(cm^2 s sr MeV)</Units>
- <FillValue>-1E+31</FillValue>
- <Measured>
- <Particle>
- <ParticleType>Electron</ParticleType>
- <ParticleQuantity>Flux</ParticleQuantity>
- <ParticleQualifier>Differential</ParticleQualifier>
- <EnergyRange>
- <Low>0.062</Low>
- <High>0.102</High>
- <Units>MeV</Units>
- </EnergyRange>
- </Particle>
- </Measured>
- </PhysicalParameter>				SPASE 1.2

## SPASE Benefits

- SPASE data model is now rich and detailed enough for us to consider using it for our primary metadata at ASC and SAMPEX RA
- This is VERY IMPORTANT – we do not have resources to maintain metadata in several forms
*SPASE can also be used as the basis for a standard message protocol for value-added services*

## Archive  VO Issues: Metadata

- So, we have the metadata issue licked. Let’s move on!
- But no... 
- SPASE 1.2 is not yet quite released – don’t feel comfy committing to writing to it just yet.
- We’ve made it considerably richer, now we have to live up to that

## Archive  VO Issues: Metadata

- And not only that, but...
- May need to generate granular-level SPASE metadata as well as product-level metadata, to be a full VO-player
- This capability would need to be retro-fitted to our existing legacy data system

## Archive  VO Issues: Data Granules

- “It’d be so nice if you’d just provide daily files (preferably in CDF format)”
- 			- some VO person...
- But VOs are just a thin coating of glue over existing systems, right?
- Reveals a dubious assumption that all relevant data exists as nice, finely chopped-up static files

## Static Data Files 

- vs
- On-The-Fly Data Subsetting Services

- Current VHO implementation:
- Returns pointers to data files satisfying a users query parameters
- Works fine when data exists as static files with the granularity expected by the VHO (daily files)
- Incompatible with ACE/SAMPEX Data Center architecture

## ACE/SAMPEX Data Center Architecture

## It Gives Our Users What They Want...

## Proposal to VOs:

- Stop worrying about the format and granularity of the data files in my archive.
- Let me instead Serve you the actual data subsets requested by your users, Not the files within which those subsets reside.
- We’ll need a standard VOArchive query language (or is it a messaging protocol?)

## ASC Data Service Architecture

**ACE Science Center**

**VO**

**Legacy API**

**URL to Data**

**Legacy**
**Web**** ****Server**
**(form interface)**

**XML Parser**

**SOAP Server**

**Discovery**
**Query**
**Handler**

**Data Query**
**Handler**

**SPASE**
**Product**
**Descriptions**

**ACE**** ****Data**
**Archive**
**(HDF Files)**

**Product info**

**Query**

**Reply**

**Data**

**URL to Data**

**New **

**Existing **

**Web**** ****Server**

**USER**

**Temp**
**Storage**

- Need Standards
- For This Part