# Presentation: 125898

## Astro-E2 Archive

- Lorella Angelini/HEASARC

## HEASARC responsibilities

- Populate and maintain the Astro-E2 science and calibration data archive
- Provide the following services
- Public access to the Astro-E2 archive
- Science data analysis software environment
- Web server  for the Guest Observer Facility
- Database system for the Astroe-E2 tables
- Long-term archive
- Astro-E2 data archive present at HEASARC and DARTS (ISAS/JAXA)

## Archival Data 

- Data sets  part of Astro-E2 archive
- Science & housekeeping  data organized by observation ( each containing Level 1 & 2 & 3 for all three instruments)
- Monitoring and/or Trend data organized by type
- Calibration data using  CALDB infrastructure
- File format:
- Science, housekeeping and calibration data are  provided in FITS format (OGIP standard)
- Preview of  summary products in other standard formats  (e.g.  GIF, JPEG)
- HTML used to record processing

## Archival Data 

- Database tables
- Allow users to select data by querying these tables via the Web based search facility (Browse)
- ASCII files using the TDAT layout (HEASARC standard)
- Record high level information  related to the data set (position, time,  and others parameters)
- Tables are :
- Master  & Instrument configurations
- Linked via the observation Id parameter
- Containing  parameter to link to the data file

## Time of delivery

- Data from standard processing
- Database  tables  master & the  instrument configuration  are updated daily
- Observation and Monitoring data also daily
- Populate the archive from  the start of  the mission
- Data during PV phase and proprietary period are kept encrypted.
- Data volume estimated up to 2 Gbyte/day

## Operational  Plan (1)

- Re-use existing software and infrastructure
- to minimize costs and risks
- maximize inter-operability with other missions
- Archive procedure fully automated (Re-use software established for Swift)
- Transfers from the processing site to the archive via DTS (Data Transfer protocol) initiated at the processing  site
- At the receiving site data are  pulled  into a staging area are validated and send to populate the archive via DAS (data archive system)
- Data and tables  on-line on a time-scale of minutes

## Operational Plan (2)

-  Archive verification between data at HEASARC and DARTS to ensure serving the same data (procedure re-use from Swift)
- Ingest calibration data  in CALDB
- Deliveries and updates  provided by the GOF
- Import Astro-E2 software into the HEAsoft package for distribution
- Deliveries and updates provided by the Software Team via CVS
- Backups
- Use RAID arrays as disk storage
- Redundant hot-backup disk system
- Backup weekly
- Backup copy to the NSSDC

## Data Access 

- Data by observation :
- HEASARC Browse interface
- Search via database tables by coordinates, time, and other parameters.
-  Data retrieval via tar file or download script to run on user machine
- FTP access
- Monitoring data
-      - FTP access
- CALDB
- via FTP to install CALDB file on user local machine
- Remote access via CALDB interface software for files used in the  reduction software

## Initial Mission Phase 

- During the PV Phase data access to the SWG team
- Data in the archive are encrypted and key distribute to the Team .
-  GO data
- Data in the archive encrypted and password distributed to the GOs. One year proprietary period
- First public data foreseen  ~ 1  year into the mission
-     (to be confirmed)
- End of the proprietary period data are decrypted at the archive site.
-  HEASARC and DARTS  needs to coordinate this operation

## Schedule

**April-May 2005 : **
- Testing data delivery from US processing site to HEASARC and archive ingest
- Test of database ingest
**June 2005 **
- Archive operational. Capability to fully ingest data in the archive and tables in the DBMS
**April 2005 **
- First pre-launch software distribution
- Coincide with the HEAsoft main distribution
- First CALDB distribution to work with software