Constructing the Subaru
Advanced Data andAnalysis Service on VO
Yuji Shirasaki on behalf of ADC
National Astronomical Observatory of Japan
Astronomy Data CenterContents
 Subaru Telescope and Instruments
 Current Status of Subaru Data Archives– STARS, MASTARS, SMOKA On-going and planned improvement ofSubaru Data Service– Quality Assessment System (NAQATA)– GRID computing system– Subaru Data Service on JVO

Related Papers
 (P2.16) Development of Quality

Assessment System for Subaru Data
Fumiaki Nakata et al.
 (P3.05) New features of Subaru

Telescope Science Archive System,SMOKA
Motohiro Enoki et al.

Subaru Telescope
SubaruTelescope isan opticalinfrared 8.2 m
telescope atHawaii,
operated by
NAOJ.

Instruments for Subaru
 CIAO OHS/CISCO
 COMICS FOCUS IRCS MOIRCS SuprimeCam
 HDS
 AO/CIAO

MOIRCSInstruments’ Field of View

Total amount of public data
as of 2006-10-08• More than 90% of all the requests
are for SuprimeCam• Current Priority Issus:
How to improve the usability ofthe SuprimeCam data

Subaru Data Archive
SkyNode
JVO
Lease LineMASTARS
18 monthSMOKA

Mitaka AdvancedSTARS

Subaru Mitaka OkayamaKiso Archive

Poster 3.05
by M. Enoki et al.

STARS
Subaru Telescope
Archive System

STARS: a sub-system of the Subaru
Telescope, not public
MASTARS: a mirror of STARSSMOKA: public archive system
JVO: Virtual Observatory Web Portal

Astronomy Data Center of NAOJ
Y. Mizumoto (JVO), M. Ohishi (JVO), S. Ichikawa (SMOKA),
T. Takata (STARS, SMOKA), Y. Shirasaki (JVO),
M.Enoki (SMOKA), M. Tanaka (JVO), S. Kawanomoto (JVO),
F. Nakata (SMOKA), A. Yoshino (SMOKA),
Y. Yamada (SMOKA)
M. Yagi Optical and Infrared Astronomy Division, NAOJ (STARS, SMOKA),
M. Ideta Center for Computational Astrophysics, NAOJ (SMOKA),
T. Horaguchi National Science Museum (SMOKA),
T. Ozawa, Misato Observatory (SMOKA)
S. Honda Optical and Infrared Astronomy Division, NAOJ (JVO)
Fujitsu Corporation (STARS, JVO)

https://stars.naoj.org/
http://sb2002.sb.nao.ac.jp/● require a user account on theSubaru computer system● science, calibration, andenvironment data
● data are copied to the user scratch
area on Subaru computer system● three query modes:– data transfer mode
– detailed query mode
– tape retrieval modehttp://smoka.nao.ac.jp/● no account is required for searching on the archive● an SMOKA account is required for data retrieval● data types:– RAW data
– WCS corrected data (SuprimeCam)– flat-fielded data (SuprimeCam)● data are transferred by FTP, DLT or DAT● Various query modes:
 Simple Search Advanced Search Calendar Search
New SUP Search (SuprimeCam)
New All Keyword Search (SuprimeCam) Full Text Search (SuprimeCam)
NewCurrent Priority Issues● data search based on the quality parameters– weather information, seeing, limiting magnitude…
–Quality assessment system (NAQATA)● providing processed data
– reduced image/spectrum, co-added image, object catalog,spectrum line parameters…
– Control the data quality– Researcher can devote himself to scientific analysis– Process the data at locations near (in term of network) thedata archive● VO interface– Interoperable data service

Quality Assessment System (NAQATA)
F. Nakata et al. Poster 2.16
 FITS format check– Check FITS standard compliance FITS header information check– Consistency check among the header values– Check if mandatory information is property written– Check format of all the header values

 Data quality evaluation (SuprimeCam)– Photometric zero point– PSF and elongation– Trend analysis of bias and background level– Monitor of gain and readout noise of CCD

PSF distribution

CCD bias variation

zero point magnitude

CCD readout noise
variation

Necessity of the Quality Control–

Raw data reduction  knowledge aboutcharacteristics of the detector is indispensable–

Information on the environment  essential toderive a physical quantity correctly–Archive users often have no knowledge of them remove the detector and environment dependency.

–The quality of the data is slowly improved as afunction of time (experience)  The reduction
procedure at each release should be referable, andthe data should be reproducible–Data provider should owe the responsibility tomanage the reproducibility of the released data

Grid Computing
 Subaru data reduction pipeline system– Web service based GRID
– Generates calibration data (Super-Flat) for all theSuprimeCam data in a week
– On-the-fly data reduction with the pre-prepared Super-Flat– Users can access to the data reduced with the most recent
algorithm– Also support the older version of pipeline

 Backend computing system of the JVO– catalog generation– photo-z calculation– Image arithmetic– …

Components of the Grid system● Computing resource management– Monitoring and Discovery Service (MDS)● Data transfer– FTP or HTTP get● Remote Job execution– Web service (Tomcat + AXIS)● Authentication– No authentication at resource by resource● All the components were developed byourselves– Plan to introduce NAREGI GRID middleware

MDS Interface
Computing Resource  MDSvoid reportStatus(String hostId,double load, int njob)
void reportJobStatus(String hostId, int jobId,String status)

Grid Client  MDS
ServiceInfo resolveService(String serviceId)
String getJobStatus(String hostId, int jobId)…

Job Execution and Data Transfer Interface
Grid Client  Computing Resource
int submitJob(String command, String argv)
String getResultURL(int jobId)
String query(int jobId)
String finalize(int jobId)

Grid Client  Storage Resource
Int copyAsync(String src, String dest)
Void copy(String src, String dest)
Void finalize(int jobId)e.g. Subaru Grid Pipeline Arch.
1. search anavailable MDS ServerserverClient

Data SearchServiceL

it aURbmu ltsures2.

5.

query status
reportstatus

3. query a location of datajob

6. Submit data transfer job

Data Analysis Service
4. get data
7. retrieve
the result

Data Storage
ServiceSuprimeCam Response Calculator

MDS Service Snap Shot
Host NameHeart Beat Status
Load Average

Number of Submitted Job

Japanese Virtual Observatory (JVO)
http://jvo.nao.ac.jp/portal
 VO portal service– Federation of the Distributed VOServices– Data Analysis Environment Subaru Advanced Data Service– Data reduction through the Webinterface– Subaru Data Service
interoperable with any VO service

Overview of the JVO Portal Service
LDAP解析
解析
解析Analysis
Servers

MDS

Web Service
HTTP Get
FTP
LDAP
P. Reg.

In

JVO
voke

Se
Di rvic
sco everyGrid
SystemAuth.
ers
UR

ste
equ
SDSS

InvokePortalUser
Storage

S. Reg.

SkyNode1.0…

InvokeSubaru2MASSData Service

Other VOs
ata gdta tin RegistryeM rves
HaQSO

P. Reg.

SIAP/SSAP

SkyNode0.7.4

1. Search a data service by keywords

2. Select a Table

3. Specify a Search Region

B-VVabs

Distributed Database Query by JVOQL• Cross match between the Subaru and Spitzercatalog• Cutout SuprimeCam images

MAG_BLog(F_IR)

– Reduction 10s(Bias+Flat+Distortion+Astromeotyr)– Transfer 30s

Summary Functionality of searching SuprimeCam data

based on the data quality indices isimplemented on SMOKA SMOKA has started to provide reduced dataof SuprimeCam since Sep 2004
 Subaru reduction pipeline system based onthe GRID architecture is under construction.
 Subaru data analysis environment will be
provided on the JVO portal.