# Presentation: 186997

## HEASARC Status and Plans

- Nick White

## The HEASARC Charter

*Established December 1990*

- Maintain and disseminate data from previous and concurrent high-energy astrophysics missions
- Provide software and data analysis support for these data sets
- Maintain and provide the necessary scientific and technical expertise for the processing and interpretation of the data holding
- Develop and maintain multi-mission analysis and support tools
- Provide catalogs of observations and ancillary information for the data holdings
- Coordinate data, software and media standards with other astrophysics sites
*Support education and outreach activities in high energy astrophysics* (not part of original charter)

## The HEASARC and High Energy Astronomy Missions

- HEASARC provides the **multi-mission infrastructure** that is used by parallel GOFs and science support centers in the LHEA and elsewhere

- Swift

- Chandra data@CXC

- HETE2

- 2

- GLAST

- Integral

## HEASARC Science Personnel

- Nicholas White: HEASARC Head
*Steven Murray: HEASARC Deputy Head (at CXC)*
- Lorella Angelini: Data Restoration, Image and timing analysis, BeppoSAX, Swift (inc. AKBAR), Astro-E2
- Keith Arnaud: Data selection software, spectral analysis
- Michael Corcoran: Calibration, GLAST, HETE-2, ROSAT
- Stephen Drake: Catalogs, website & archive quality assurance,  EUVE, RXTE, CGRO
- James Lochner: Education and public outreach
- Thomas McGlynn: Archives, Virtual Observatory, SkyView
- Bill Pence: FITS, Software, HEASOFT, Hera

## Recent HEASARC Highlights

- Hardware:
- Acquisition of 8.4 TB (12 TB raw) TB network appliance (March 2003)
- Upgrade to Gigabit network (Oct 2003)
- Science Archive:
- XMM downloads now 2nd in volume to RXTE
- HETE-2 archive reaches 150 GB and 51 bursts
- Specialized Swift Browse interface
- Software:
- HEAsoft 5.3 release (November 2003): Improved fv integration with DS9
- Browse 7.0 release (January 2004): new upload & download features
- XSPEC 12 testing by Integral SPI team (ongoing)
- Catalogs:
- GRBCAT master catalog of gamma-ray bursts (Feb 2004)
- Education and Outreach
*SkyView* SciAm Top 50 award (June 2003: 2nd year in a row)
- Teacher’s Hera beta implementation (July 2003)
- Services:
- First VO Tool: Data Inventory Service (July 2003)

## The Physical Archive

- _Active Missions_
- RXTE (1995- )
- Chandra (1999- ) [data at CXC]
- HETE-2 (2000- )
- Integral (2001-)
- XMM-Newton (1999- )

- _Past Missions_
- Ariel 5	EXOSAT
- ASCA 	Ginga
- BBXRT	BeppoSAX
- CGRO	HEAO 1
- Copernicus	HEAO 3
- COS B	OSO 8
- DXS 	ROSAT
- Einstein 	SAS 2
- EUVE 	SAS 3
-  	Vela 5B

- •	Data from 24 missions currently in the archive
- •	355  astronomical catalogs & mission tables
- •	The archive volume was
- 3250 Gigabytes as of the end of 2003

- _Upcoming Missions_
- Swift (2004 Launch)
- Astro-E2 (2005 Launch)
- GLAST (2007 Launch)

## Usage & Data Statistics

**        ****Gigabytes transferred per year**
_**	**__**1998**__**	**__**1999**__**	**__**2000**__**	**__**2001**__**	**__**2002**__**	**__**2003**_
**Datasets by**
**ftp & http****	**844	1406	1880	2270	2917	3943
**Science Web**
**Pages & Images****	**198	 234	 371	  399	  580	  872
**Main E&PO Web****	**
**Pages & Images****	**203	 358	 678	1010	3108	7581

## Data Transfers by Mission over Time

## Current Web Service Statistics

- Archive volume: 3.3 TB (mostly compressed)
- Current ingest rate: 400 GB/yr
- End of 2004 anticipated rate: 1.4 TB/year, with increased or new flows from Swift, Integral, and XMM
- Missions in archive:
- Ariel-5, ASCA, BBXRT, BeppoSAX, _*Chandra*_, Compton, Copernicus, COS-B, DXS, Einstein, EUVE, EXOSAT, Ginga, HEAO-1, HEAO-3, *HETE-2*, OSO-8, ROSAT, *RXTE*, SAS2, SAS3, Vela 5B, *XMM-Newton*
- Downloads (annual): 3.9 TB of data (by FTP and HTTP), 0.9 TB of science web pages (by HTTP) and 7.6 TB of E&PO web pages
- Browse: 355 local tables (+ ~3000 VizieR); 2,150,000 queries
*SkyView: *>60 surveys; >1,000,000 images generated
- 26,000 bibliography entries linking ADS to HEASARC datasets (ROSAT, ASCA, XMM-Newton, and some RXTE datasets)

## HEASARC Hardware Configuration

- HEASARC
- (load balancer)

- HEASARC1
- 2 GHz Dual Processor

- HEASARC2
- 2 GHz Dual Processor

- More slots available

- Web servers

- Safehouse

- 6 TB
- Assorted
- Storage

- Online archive backup

- Hera (x5)
- 1 GHz

*SkyView* (x2)
- 1 GHz

- 150 GB
- Surveys (x2)

- 240 GB
- External
- Survey Cache

- 400 GB
- User
- Cache

- GSFC Portal

- Linux, Solaris, Mac, Windows, ...

*Gigabit*

*100 Mb*

- Database (Sybase)
- 2 GHZ dual Processors

## Web site maintenance

- Main HEASARC web site has 15,000 static pages and hundreds of dynamic pages (excluding the FTP area and data archive) comprising 7 GB in total
- Ensuring valid and up-to-date links pages requires constant attention
- In the past, invalid links left unattended grew to ~500.
- Concentrated effort has now eliminated all but a few broken or out of date links (Newly broken links are fixed weekly)
- HEASARC is now 99% Section 508 compliant
- Assuring appropriate access to HEASARC Web pages involved changes to most Web pages
- Compliance changes were often beneficial for all users.
- Maintenance cost of 508 compliance is <=0.05  FTE effort

## Browse Catalog and Archive Interface

- Capable of querying all local and VizieR catalogs
- Local and remote archives (e.g., CXC)
- HTML, text, VOTable, FITS and Excel ouputs
- New capabilities:
- Expressions allowed in search criteria ( fluxX/fluxY > 2)
- Better integration with *SkyView*
- Upload files of targets (in Browse 7.0)
- Download queries for later re-use (7.0)
- Dataset ID info added when available (7.0)
- Links between tables

## Current Active Missions

- Chandra
- Browse service fully linked to CXC archive
- RXTE
- Dominates current archive downloads in terms of GB
- Simple XTEIndex table developed and deployed (more simplification needs to be done)
- HETE2
- Some HEASARC-written FITS products for GRBs
- Difficult to use most data delivered to archive
- Similarities to data recovery efforts of early HEASARC
- XMM-Newton
- In 2003, second biggest HEASARC mission in archive volume & downloads
- Archive contains both US & non-US Guest Observer data
- Integral
- Use modified version of HEASARC archive software at ISDC
- HEASARC Browse can link to ISDC Browse tables
- Archive at HEASARC will open in July 2004

## Preparations for Swift

- New HEASOFT package
- MOU delineating HEASARC responsibilities
- End to end testing including archive ingest.
- Customized mission interfaces:
- Individualized interface to Browse
- HTML wrappers for standard Browse scripts
- Links between Browse tables
- AKBAR: new service to import external information from observers
- HEASARC responsible for some high-level data products – e.g., overall burst evolution
- Mission requires rapid dissemination of data

## Swift Browse Interface

## Preparations for GLAST

- PDMP review
- Consultations on archive and catalog design
- Review proposed FITS formats

## Standards

- FITS
- Subsumed GSFC FITS Office Web site
- Develop and coordinate domain conventions for high energy data (OFWG))
- Organize periodic HEADCC meetings
- FITS Libraries Support
- CFITSIO is essentially reference implementation of FITS
- HEASARC also supports widely used FITS libraries in Java and IDL.
- New C++ object implementation based on CFITSIO
- Standard Dataset IDs
- VO Standards
- Browse support for VO Cone search
*SkyView *support for VO Simple Image Access Protocol
- VOTable (and FITS) outputs available through Browse
- VO Service Metadata descriptions for HEASARC resources
- Publication of metadata using Open Archives Initiative (OAI) protocol

## Software Tools and Services

- HEASARC software is frequently downloaded
- fv (FITS browser and editor): 500 downloads/month
- CFITSIO library: 500/month
- HEASOFT: 200-300/month
- HEASOFT
- New packages for Swift and Astro E2 missions under active development
- Use HEADAS in order to provide a simple platform-independent procedure for new software builds and updates
- XSPEC New major release (12.0) in alpha test
- Complete re-write of Xspec internals in object oriented fashion
- Can support coded aperture mask data sets

## Hera

- Provides a full, on-line analysis environment to users.
- Directly connected to archive, no downloads needed
- Up to date versions of HEASOFT tools automatically available
- Easily upload user data and download results.
- Uploaded data screened to ensure service is not used inappropriately.
- Uses standard HEASARC FV FITS browser tool as user interface.
- Why?
- Large volume of data (e.g., Integral, RXTE) downloads and/or software installation may be infeasible for some users
- With Hera, users can start using HEAsoft and other software without the need for local installation
- Status
- Initial system operational and being used actively.
- Proposal for continued funding has been submitted
- Both teacher and student interfaces in beta testing

## New and Updated Catalogs

- ~24 new catalogs or major catalog updates in last year.
- Major correlative catalogs: VERONCAT, GB6, etc
- High energy source catalogs:  XMMSSC, BMWHRICAT, INTREFCAT
- Master catalogs updated regularly
- GRBCAT: Compilation of all gamma-ray bursts
-  VizieR connection for transparent access to massive catalogs (2MASS, USNO B, GSC 2)

## Education and Outreach

- Education Web Sites:
- Imagine the Universe (Grades 6-12): 500K hits/month.
- StarChild (Grades 1-8) [in collaboration with L. Whitlock]: 1.25 M hits/month.
- New poster on origin of the elements
- X-ray analysis school
- Highly successful and oversubscribed
- Host for APOD Web site
- StarChild/Imagine/APOD CDs

## Other Projects

*SkyView*
- Java redesign
- New surveys  (DSS2, Halpha, SFD) & SDSS soon.
- Request for continued funding submitted
- ClassX
- Automated classification of entire RASS and WGACAT samples of X-ray sources using supervised learning algorithm

- Density of source requests in* SkyView*

## Virtual Observatory Involvement

- Data Inventory Service
- HEASARC metadata inventory
- Standardized metadata describing all HEASARC tables,data file types, surveys and services.
- Registered with VO to enable effective discovery of HEASARC resources.
- Table metadata
- HEASARC support for VO standards

## Data Inventory Service

- Access to distributed services that use VO protocols.
- Many sites contributing data
- Dynamic service lookup using VO registries
- Catalog, observations, Images
- _[http://heasarc.gsfc.nasa.gov/vo/](http://heasarc.gsfc.nasa.gov/vo/)_** **

## Near Term Missions and Deadlines

- Launches
- Swift: Fall 2004
- Astro-E2: Early 2005
- GLAST: Early 2007
- HEASARC Senior Review Proposal Due: March 17, 2004

## Current and Upcoming Issues

*Science*
- May need greater emphasis on supporting  gamma-ray astronomers as Integral and GLAST Archives come online and become an increasing fraction of the HEASARC’s archive
- Must deal with ever increasing complexity of data and calibration
- Coded aperture masks (Swift, HETE2, Integral)
- Complex source dependent response (“ + GLAST)
- Multi-satellite observations (Con-X, LISA)

## Current and Upcoming Issues

*Hardware*
- CPU and disk space needs achievable, but bandwidth is a potential limitation
- Datasets may be growing faster than bandwidth
- Demand for data will spike after GRB detections
- Higher speed linkage to Internet possible but costly
- ... and doesn’t help users who have slower connections
- Move towards more user processing at HEASARC
- Hera, grid computing
- On-line backup for archive is presently in same building as primary HEASARC archive.
- Planned move of backup archive to another building when space and network is ready (Summer 2004)

## Current and Upcoming Issues

*Local Software and Services*
- Integration of tools and services
- Strengthen SkyView/Browse, Browse/NVODIS, Hera/Browse, ... cross-links.
- Enable coupling archive/web services and HEASOFT tools.
- Systematic attention to internal documentation of HEASARC systems