Recent Progress in Building the
Virtual Observatories in the World
Masatoshi Ohishi / NAOJ & Sokendai
大石雅寿 / 国立天文台 & 総合研究大学院大学
masatoshi.ohishi@nao.ac.jp

July 19, 2006

NAOC China-VO Seminar

1

Data Resources in the World• ESO VLT @ Paranal• ESO telescopes @ La Silla• Telescopes in Mauna Kea
Subaru, Keck, JCMT, CSO, SMA,,,• Telescopes on board satellites
HST, GALEX,,,,

July 19, 2006

NAOC China-VO Seminar

2

Data Resources in NAOJ
• Subaru 8.2m Optical-Infrared Telescope• Kiso 105cm Schmidt Camera

Nobeyama 45m• Okayama 188cm Optical Telescope• Nobeyama 45m Radio Telescope• Nobeyama Millimeter Array• Nobeyama RadioheliographSubaru• VSOP• VERA• ALMAJuly 19, 2006

NAOC China-VO Seminar

3

Data Resources in JAXA/ISAS• ASCA X-ray astronomy satellite• YOHKO solar physics satellite• Ginga X-ray astronomy satellite• HALCA VLBI satellite• Geotail geomagnetosphere satellite

ASCA• Akebono aurora observation satellite• ASTRO-F Infrared satellite• ASTRO-E2 X-ray satellite
YOHKO• SOLAR-B

July 19, 2006

NAOC China-VO Seminar

HALCA4

ALMA Project© JulyNAOJ19, 2006

NAOC China-VO Seminar

5

Data Productivity• Nobeyama Radio ：~1TB/yr• Subaru@Hawaii ：~20TB/yr• ASTRO-F(will be launched)：several 100 GB in total• ALMA（planned） ：~PB/yr
Flood of excellent data (survey data)
Digitized & Archived
July 19, 2006

NAOC China-VO Seminar

6

Astronomy based on Archives– large fraction of astro-papers based onarchives– HST archive use growing faster thanarchive
30

Gbytes/Day25
2015105

AlreadyAlreadymoremoreretrievalretrievalthan
thaningest!
ingest!

01994.8

1995.3

1995.8

1996.3

Ingest1996.8

Year
July 19, 2006

1997.3

1997.8

1998.3Retrievals
Ingest1998.8NAOC China-VO Seminar

1999.3graphics fromUS NVO 7projectToo hard to collect andanalyze whole data.
Need Cat’s help

Data Rate

Accelerate astronomicalresearch, and sufficient
time for research itself !! Virtual Observatory

July 19, 2006

Nobeyam Radio Obs.

~ 1 TB/yr

SUBARU telescope~ 20 TB/yr

ALMA~ 1 PB/yr

Even for educational use

NAOC China-VOAccessible
Seminarfrom anywhereat any time8VO– New Research Infrastructurein the 21st Century
A collection of integrated
astronomical data archives andsoftware tools that utilize
computer networks to create anenvironment in which research
can be conducted.
http://www.encyclopedia.com/html/v1/virtobserv.asp
July 19, 2006

NAOC China-VO Seminar

9

International Endorsements• IAU XXVth GA Res. (2003 Jul.)• OECD Rec. (‘04 Aug)– place archives that may be accessible
via internet– provide adequate funding as long-term
issuesJuly 19, 2006

NAOC China-VO Seminar

10

IAU XXVth GA Res. (2003 Jul.)
Recommends
1.
that, data obtained at major astronomical facilities should, after areasonable proprietary period in which they are available only toobservers or other designated users of the facility, be placed in
an archive where they may be accessed via the internet by all
research astronomers. As far as possible, the data should beaccompanied by appropriate metadata and other information ortools to make them scientifically valuable,
2.
that, such data should not be subject to intellectual property
rights. The form in which data are made available, and thesubsequent processing of such data, may be appropriately
protected by copyright laws, but the fair usage (includingeducational purposes) of the archive data themselves should not
be subject to restrictions,3.
that, funding agencies provide encouragement and support toenable data produced by astronomical research that they fund to
be deposited, after some proprietary period as defined above, inrecognized data archives which provide unrestricted access tothese data.
July 19, 2006
NAOC China-VO Seminar
11

OECD Rec. (‘04 Aug)
Recommendations:

New projects and facilities must take the datamanagement, storage, maintenance, anddissemination into account at the earliest
planning stages, consulting potential users in
the process. Agencies and governments
should consider adopting the IAU resolutions
as the basis for progress in this field.
Agencies should recognise that this is an
important long-term issue and should coordinate plans, provide adequate funding on
a long-term basis, and support development
and maintenance of the needed infrastructure.
Agencies should encourage broadening ofexisting VO collaboration into a fully
July 19, 2006
NAOC China-VO Seminar
12
representative global
activity.

VO Projects in the world• 15 countries and region• International Virtual Observatory Alliance (IVOA)
Standards to interoperate VOs• Japan – Language to accessfederated DB

July 19, 2006

NAOC China-VO Seminar

13

July 19, 2006

NAOC China-VO Seminar

14

Standardization in IVOA
• Query language to distributed DBs (VOQL)• Meta-data：contents, protocol to interchange
based on OAI-PMH• Protocols to retrieve images, spectra, and so on
SkyNode, SIAP, SSAP, STC, etc.
• Unified attribute names in DBs
UCD (Unified Contents Descriptions)• Output Format：VOTable (XML)incorporates FITS
• etc
July 19, 2006

NAOC China-VO Seminar

15

IVOA Interoperability WS• Twice a year
• Discussions
towardstandardization• Humannetwork as abasis for cyber
network

July 19, 2006

NAOC China-VO Seminar

16

Interconnected VO Data services inthe World

Canada VO

July 19, 2006

NAOC China-VO Seminar

17

July 19, 2006

NAOC China-VO Seminar

18

Software Tool providers• VO-China– VOFilter : VOTable 
OpenOffice data
– VOIMPAT : Image
processing andanalysis tool– ,,,• VO-India– VO-Plot: to plot
contents in VOTable– ,,,July 19, 2006

NAOC China-VO Seminar

19

Workflow Description Language
Based on BPEL4WS••

Variable definition
Controls (Loop,Condition)• Parallel execution• Invoke externalservices• Invoke built-in JavaClasses
July 19, 2006

NAOC China-VO Seminar20

Management of Multiple Servers• Data Analysis services under JVO– SExtractor (Image → Catalog)– HyperZ (Catalog → photometric redshift)• In operation on multiple servers→ Monitor and Discovery Service (MDS) server

JVO Portal

MDS Server
Service Request

ExecuteAvailable Server Info
Confirm Job Completion
Job Completion Info

# of Jobs
CPU load
Job status

Analysis Servers

Analysis ServerManagement Data

Job submission
ClientResults

Parallel Executions on Multiple Servers are available
July 19, 2006

NAOC China-VO Seminar

21

Schematic diagram of VOs
Web Services

Input Queries and Analysis
Parameters

Web Services

Resource Meta-data

Web Services

User
Interface・Query Editor
・VOTable viewer
・DB queries
・Plotter・etcController

SIAP/SSAP/SkyNode・parser
・scheduler
・Service caller

User

US/CAN

SIAP/SSAP/SkyNode

Web Services
EuropeJuly 19, 2006

NAOC China-VO Seminar

JVO Portal Server22

JVO Query Language (JVOQL)JVOQL is designed as a prototype of VO Query Language.
Characteristics of the JVOQL:• SQL based Query Language• Query Language for the distributed astronomical DB.
• Can search and retrieve observational data as well ascatalog data
• Upward compatible with the ADQL and SIAP syntax.
• Scalable syntax structure. Very simple core syntax andextension syntax packages.
July 19, 2006

NAOC China-VO Seminar23

JVOQL Syntax Requirement 1• Unified query language for both the catalog and observation
data such as image data, spectrum, 3D-cube, photon list …
– Parameter query (SIAP) can be replaced by SQL thinkingthat the parameters are columns of a relational table.

– Observational data or pointer (URL) to retrieve the datais also a column of the relation table.
http://jvo.nao.ac.jp/imageData?Pos=24,5&Size=0.2&format=VOTable
SelectFrom
Where

imageURL, …
naoj:imageData
pos=Point(24,5) and size=0.2 and format=‘VOTable’“pos” , “size”, “imaegURL”
are virtual columns.
NAOC China-VO Seminar

July 19, 2006

24

JVO SkyNode Architecture
JVO Portal• Apache Axis and Tomcat are used forimplementing the VO standard interface, andGlobus Toolkit is used for an optional Gridinterface.
• Four kinds of query languages are converted tojava class SelectSQL and query is executed
through JDBC-like interface.

Internet• Query results is obtained as ResultSet, andtable data is formatted to VOTable or CSV file.

JVO SkyNode

JVOQL

JVOQL
ADQL-xSIAP
SSAPJuly 19, 2006

Grid
GTK 3 (optional)
HTTP/SOAP
AXIS

Translator
Query Executer
Formatter

HTTP/Parameter
Tomcat
NAOC China-VO Seminar

SelectSQLJDBC forSkyNode
ResultSet
getMetadata()
getString()
getFits()
getJpeg()

JVO
SkyNodeDBMS25

Development of JVO Skynode Toolkit
JVO Skynode :• can accept ADQL-x over HTTP/SOAP, SIAP overHTTP/Parameters  VO compliant,• can accept JVOQL over HTTP/SOAP and Grid(experimental) .  functionality test of JVOQL,• returns VOTable, CSV file and FITS file,
The JVO Skynode toolkit is intended to be used as anwrapper for existing data services to become VO compliant
 easy and quick implementation of the skynode interface
on the existing system.
July 19, 2006

NAOC China-VO Seminar26

Resource Metadata
service metadata
content metadata

Identity metadata
curation metadata
title
short_nameidentifier
publisher
publisher_id
creatorcreator_logo
contributer
dateversioncontact_namecontact_email
service_interface_url
service_base_url
service_http_result
service_standard_uri
service_standard_url
service_msr

stringstringURI
stringURI
stringURL
stringstringstringstringe-mail addressURL
URL
MIME type
URI
URL
float,decimal degrees

ucd
unit
datatype
width
precisionarraysize

stringstringstringint
stringstringcatalog
table
columnidentity○○○

July 19, 2006

curation○○○

service○○○content○○○

column××○

column metadata

subject
description
sourcereference_url
type
coverage_spatial
coverage_region_of_regard
coverage_spectral
coverage_spectral_bandpass
coverage_spectral_central_wavelength
coverage_spectral_minimum_wavelength
coverage_spectral_maximum_wavelength
coverage_temporal_start_time
coverage_temporal_stop_time
coverage_depth
coverage_depth_unit
coverage_object_density
coverage_object_countcoverage_sky_fraction
resolution_spatial
resolution_spectral
resolution_temporalcontent_level
facility
instrument
formatright

NAOC China-VO Seminarstring,liststring,free text
stringURL
string,list
stringfloat,decimal degrees
string,liststring,listfloat
float
float
stringstringfloat
string
float
int
float
float
float
float
string,liststring,liststring,liststring,liststring

27

Catalog、Table Resourceはcolumn metadata 以外を持つ。

Exchange of Meta Data:OAI-PMH
Searchable Registry
Publishing Registry
Publishing Registry

Query Services
ouso varitsse
Acc

Data
Servicees
servicAnalysis
Service
Virtual Observatory Client

July 19, 2006

NAOC China-VO Seminar

28

JVO is seen from the UK VO

July 19, 2006

NAOC China-VO Seminar

29

Elapsed time to querying US VOs
wavelengthSurvey
nameservertime(sec)

X-ray

Chandra cda.harvard.edu

1.715

Infrared 2MASS

mercury.cacr.caltech.
edu

3.536

Radioadil.ncsa.uiuc.edu

7.115

VLABest Records -- Some servers may be overloaded from time to time
July 19, 2006

NAOC China-VO Seminar

30

AstroGrid Workbench

July 19, 2006

NAOC China-VO Seminar

31

VOSpec: SSAP viewer

July 19, 2006

NAOC China-VO Seminar

32

July 19, 2006

NAOC China-VO Seminar

33

July 19, 2006

NAOC China-VO Seminar

34

July 19, 2006

NAOC China-VO Seminar

35

July 19, 2006

NAOC China-VO Seminar

36

July 19, 2006

NAOC China-VO Seminar

37

July 19, 2006

NAOC China-VO Seminar

38

July 19, 2006

NAOC China-VO Seminar

39

July 19, 2006

NAOC China-VO Seminar

40

July 19, 2006

NAOC China-VO Seminar

41

July 19, 2006

NAOC China-VO Seminar

42

July 19, 2006

NAOC China-VO Seminar

43

July 19, 2006

NAOC China-VO Seminar

44

July 19, 2006

NAOC China-VO Seminar

45

July 19, 2006

NAOC China-VO Seminar

46

Search for Gravitational Lenses
produced (?) by Cosmic Strings• SXDS data observed by Subaru• Query results were obtained less than 5 min,displaying SEDs
• It has been proventhat VO canaccelerate
researches.

July 19, 2006

NAOC China-VO Seminar47

Cosmic String Search Result

July 19, 2006

NAOC China-VO Seminar48

Study on Environment around QSOs

July 19, 2006

NAOC China-VO Seminar

49

Discovery of a Brown Dwarf：
SDSS/2MASS
discoveries likethis much easier if
databases jointly
queryable

July 19, 2006

NAOC China-VO Seminar

50Detection of MANY Type II Quasars

July 19, 2006

NAOC China-VO Seminar51

Latest Science outputs• Aug 2006 – VO Special Session @IAU GA– 5 days session• 200+ participants registered already– ~30 science talks out of ~50– ~100 posters

July 19, 2006

NAOC China-VO Seminar52

New Moves• Some VO projects move from R&D phase
to operations phase– UK, Japan, US, EU,,,• New VO projects– Brazil, Chinese Taipei, Blugaria, Tagikistan,Ireland, Czech,,,July 19, 2006

NAOC China-VO Seminar

53

Items to be done
• Distributed Storage to store
query/analysis results• Secure access to VOs : singlesign-on• Other Standardizations– Standard application interface• Advertisement to Data Centers– data centers need implement VOinterfaces• VO schools for astronomers
July 19, 2006

NAOC China-VO Seminar54VO : basis for EACOA

July 19, 2006

NAOC China-VO Seminar55

http://www.ivoa.net/

July 19, 2006

NAOC China-VO Seminar

56