# Presentation: 499078

## yourSky:  Rapid Desktop Access to Custom Astronomical Image Mosaics

**yourSky:  Rapid Desktop Access to Custom Astronomical Image Mosaics**

- SPIE Astronomical Telescopes & Instrumentation:Virtual Observatories Conference
- August 25, 2002

**http://yourSky.jpl.nasa.gov**

**J. C. Jacob, D. Curkendall, L. Husman: Jet Propulsion Laboratory**
**S. G. Djorgovski, A. Mahabal, R. Brunner: California Institute of Technology**
**J. Good: Infrared Processing and Analysis Center**
**G. Kremenek: San Diego Supercomputer Center**

- Space Science Applications of Information Technology Program

**Notes:**
Contact: 
Joseph Jacob, Member of the Information Systems and Computer Science Senior Staff, Parallel Applications Technologies Group
Jet Propulsion Laboratory, 4800 Oak Grove Drive, Mail Stop 126-104, Pasadena, CA 91109-8099
Email Joseph.Jacob@jpl.nasa.gov; Phone 818-354-0673; Fax 818-393-6141

## Outline

- Motivation
- Virtual Observatories
- yourSky (http://yourSky.jpl.nasa.gov)
-  User Perspective
-  Behind the Scenes
- Current and Future Activities

## The Data Avalanche!

**Gigapixels**

**m****2**

**Growth in**** ****Aperture**** ****&**** ****Focal**** ****Plane**
**Of Institutionally Managed Observatories**

**Notes:**
In recent years the astronomy community has witnessed exponential growth in the size of dataset that can be captured from a single observatory.
We’re now at the point where the largest surveys are at multiple TB and there exists surveys at wavelengths that span the entire EM spectrum. 
This data avalanche represents both a significant opportunity for new scientific discovery and a significant challenge: How can we effectively and efficiently extract useful information from the such a massive data store?

## Virtual Observatories

- Community driven.
- Community built.
- Community access.
- Emphasis on many interoperable components developed and deployed by domain experts in different areas.
- Highly distributed, including centers for:
- Archive
- Processing
- Visualization
- Exploit high performance computation and communications assets.

**Notes:**
To address this data avalanche problem, both NASA and the NSF are funding components of the NVO.
As a community effort, the NVO will necessarily be widely distributed with components for data archive, processing and visualization developed and deployed by domain experts in different areas.
Many of these components will require high performance computing, supercomputing, high bandwidth networks, and powerful visualization capabilities.
However the real challenge facing NVO architects is Community access.            

## Transparent Use of High Performance Infrastructure

**VO Portal**

- Supercomputers

- Huge Archives

- High Bandwidth Networks

**Remote Clients: Minimal Resource Requirements**

**Notes:**
Many astronomers have modest computing resources on their desktop, don’t have direct access to supercomputers, don’t even know how or if they can get an account on such machines. Also, supercomputers can be difficult to use efficiently and it’s not always easy to retrieve the particular chunk of data that you need from TB archives.  
Therefore, to be effective, the NVO will need to provide transparent use of these high performance computing assets and be able to both take advantage of the high bandwidth connection to these assets, but also cope with very low bandwidth network connections and light weight computing resource at the client side.

## Example: Custom ‘On-the fly’ Mosaic Portal

- Custom access = user specifies dataset, location, size, resolution, coordinate system, projection, data type, and image format.
- Architecture invites growth to expand options for custom image processing – multiple background removal techniques, overlap blending recipes, multiple surveys, etc.

**Notes:**
As one example of an NVO application that fits this architecture, we developed a custom on-the-fly mosaic portal called “yourSky”.
Our overarching goal is to provide custom access to a compute intensive service that delivers science-grade image mosaics to user’s at their desktop.
In this context, custom access means that the user specifies dataset, location on the sky, resolution, coordinate system, projection, ...
So, no matter where in the sky you are looking or how big of a region you need, you can get back just the data you want as a single FITS image. No more, no less. 
You no longer have to care about the native image partitioning scheme of the dataset you are studying.

## Why Mosaic?

- Object much larger than a single image plate.
- Object much smaller than a single image plate.
- Object falls at boundary between neighboring image plates.
- Multi-wavelength science!

**Notes:**
Mosaicking is necessary for cases where the object is much larger than a single image plate or falls at the boundary between neighboring plates. It can also be useful in cases where you are studying an object much smaller than a single plate so you need not process an entire, say 1GB, plate when the object you are studying is just a few arc minutes across.
Most exciting, mosaicking multiple datasets to the same grid enables multi-wavelength science.

## yourSky Custom Mosaic Portal http://yourSky.jpl.nasa.gov

- yourSky Custom Mosaic Portal http://yourSky.jpl.nasa.gov

**yourSky can access all of the publicly released DPOSS and 2MASS images for custom mosaic construction.**

**Notes:**
We call our custom mosaic service yourSky and the URL is shown here.
You enter your mosaic request parameters in a simple form interface shown here.
Two of the larger astronomy archives are accessible with yourSky: 2MASS and DPOSS.
After you enter your request parameters and hit submit, you eventually get back an email telling from where you can download your mosaic.

## yourSky: An Architecture for Desktop Access to Parallel Mosaic Codehttp://yourSky.jpl.nasa.gov

**yourSky**: An Architecture for Desktop Access to Parallel Mosaic Codehttp://yourSky.jpl.nasa.gov

**Notes:**
You may end up with an image mosaic such as this DPOSS mosaic of Andromeda.
But a lot happens behind the scenes to get to this point, as shown in this architecture diagram.

## yourSky Components

- Parallel Image Mosaicking Code
- Request Management
- Plate Coverage Database
- Data Management
- Retrieval of image plates from remote archives.
- Local data cache.

**Notes:**
So, we’ve seen that yourSky has components for actual image mosaic construction, 
For request management,
For determine region of coverage of the input image plates,
And for data management, including both retrieval of image plates from remote archives and maintaining a local data cache on the yourSky server.

## Mosaicking Code –Coordinate Mapping

- Input Image Plates

- Output Mosaic

- (xi,yi)

- (xo,yo)

- (RA,Dec)

- 1. Map input pixel coordinates (xi,yi) to sky coordinates (RA,Dec).

- 2. Map sky coordinates (RA,Dec) to output pixel coordinates (xo,yo).

**Notes:**
To construct a mosaic, image coordinates need to be mapped from input image space to output mosaic image space.
In yourSky, this is a two step process.  First we map input image coordinates to sky coordinates.
Then map those sky coordinates to output mosaic image space.

## Custom Coordinate System and Projection

- Coordinate Systems:  Galactic, Ecliptic, J2000 Equatorial, B1950 Equatorial.
- WCS projections: LIN, TAN, SIN, STG, AZP, ARC, ZPN, ZEA, AIR, CYP, CAR, MER, CEA, COP, COD, COE, COO, BON, PCO, SFL, PAR, AIT, MOL, CSC, TSC, DSS, PLT.

**Notes:**
In yourSky, this coordinate mapping is done using the WCS library.
We support galactic, ecliptic, J2000 and B1950 equatorial coordinate systems and all of the standard WCS projections.

## Custom Image Format and Data Type

- Image Format:
- FITS
- JPEG
- PGM
- PNG
- TIFF
- Raw Data

- Data Type:
- 8-bit unsigned integer
- 8-bit signed integer
- 16-bit unsigned integer
- 16-bit signed integer
- 32-bit unsigned integer
- 32-bit signed integer
- Single precision floating point
- Double precision floating point

**Notes:**
yourSky can generate output images in different image formats and with different pixel data types.
Of course, FITS will be used most often, but, support is also provided for JPEG, PGM, ... and Raw Data.
The Raw Data format supports all of these pixel data types, but the other image formats support only a subset of these.

## Slide 14

**Notes:**
To give you an idea how long it takes to construct a typical image mosaic, I’ve run a few timing tests on an SGI Origin 2000 back at JPL.
Here I’ve plotted wall clock time versus number of processors on the machine up to our largest user queue size of 64 processors.
The blue solid line is the result for a full 1 arcsec resolution 2x2 degree 2MASS mosaic of the galactic center, which required 174 image plates or 365 MB to produce a 207 MB output mosaic.
The pink dotted line is the result for a 1/10 resolution or 0.1 arcsec resolution 2x2 degree DPOSS mosaic of M31, which required 2 image plates or 2GB to produce a 2 MB output.
Anyway, it takes roughly a minute to produce the result on 64 processors.

## Request Management

- Requests are queued by the Mosaic Request Spooler and selected for processing by the Mosaic Request Manager.
- User’s priority is reduced according to the number of pixels generated in past 24 hours.
- A user that has had to wait for data retrieval from tape gets high priority once the data is ready.
- Ensures:
- All users get a chance to have their mosaic processed.
- No single user dominates the available resources.

## Plate Coverage Database

- Contains the minima and maxima of the longitudes (RAs) and latitudes (declinations) in each supported coordinate system.
- Query result is a list of the input image plates required to fulfill the mosaic request.
- MySQL implementation.
- Accessible as a stand-alone service: _[http://yourSky.jpl.nasa.gov/query/index.html](http://yoursky.jpl.nasa.gov/query/index.html)_.

## Data Management

- Input image plates used to construct mosaics are cached locally for later requests and discarded on a least recently used basis.
- Completed mosaics are stored in a work area for 24 hours (for user download).
- Input image plates are retrieved from:
-  **DPOSS:** High Performance Storage System (HPSS) at Caltech’s Center for Advanced Computing Research.
-  **2MASS:** Storage Resource Broker (SRB) at the San Diego Supercomputer Center.

## Future Plans

_**Montage:**_
- yourSky is the baseline code for Montage, an ESTO-CT Round 3 Grand Challenge Project (P.I.: T. Prince).
- Collaboration between CACR, IPAC and JPL.
- Montage will improve upon yourSky:
-  **Science Quality** – Flux preservation / Background matching.
** ****Performance** – Throughput
-  **Interoperability** with NVO infrastructure.
-  **Interoperability** with Teragrid infrastructure.
- Montage has staged code improvement deliverables through January, 2005.
- B. Berriman: 10:20am Mon. Aug 26, Kona Ballroom 3

## Future Plans (cont.)

_**Graphical Front End to yourSky**_
- All sky, medium resolution pan and zoom over each dataset.
- Catalog overlay capability.
- Interoperable with the yourSky server.
- Used to help formulate requests to yourSky.

## Future Plans (cont.)

_**Information Power Grid**_
- NASA’s computational grid infrastructure.
- Globus enabled version of yourSky.
- Launch yourSky mosaicking code on the Grid instead of on local machine.
- Dramatic improvements in the size and number of mosaic requests we can handle.

**Notes:**
At JPL we also have plans to deploy a globus enabled version of yourSky, which will allow us to launch the mosaicking code on the massive computing assets that are part of the Information Power Grid, NASA’s computational grid infrastructure.
We expect that this will lead to dramatic improvements in the size and number of mosaic requests that can be handled by yourSky.

## Summary

- yourSky supports large scale data access and image mosaicking on supercomputers.
- yourSky places minimal computing requirements on the users: Web browser.
- _[http://yourSky.jpl.nasa.gov](http://yoursky.jpl.nasa.gov/)_
- Questions?