2007.Oct.30-Nov.1
External Review

Overview ofILDG: International Lattice Data GridandJLDG: Japan Lattice Data Grid
Mitsuhisa Sato
Center for Computational Science
University of Tsukuba

Grid R&D Activities in Lattice QCDThe ILFTNet (International Lattice Field Theory Network) Project (FY2004〜FY2005) aimed to construct a research network among leading institutes
in Japan, USA, UK, Germany and other countries in the field of latticequantum field theories, through promotion of international collaboration, achain of workshops and seminars, and researcher exchange.
The hub institute in Japan was the Center for Computational Sciences, Univ.
of Tsukuba

ILDG is an international project to develop a grid of data grids for sharing
lattice QCD configurations world-wide. ILDG is an infrastructure of ILFTNet.
LQA is a database of QCD configurations maintained by the Center for
Computational Sciences (CCS) since Dec. 2003.
JLDG is a data-grid infrastructure for Lattice QCD (LQCD) community inJapan.
2International Lattice Data Grid Project
JSPS program 2005-2006

R& D of International Lattice Data Grid (ILDG) and international
collaboration in computational particle physicsData sharing among key cites in USA/UK/Germany/Japan etc

UK
ＵＫＱＣＤ
network
University of Edinburgh
EPCC
Fermi National Accelerator Lab

University of Tsukuba
CCSUSA
ＳｃｉＤＡＣ
networkDESY
Von Neumann InstituteＷａｓｈｉｎｇｈｏｎ ＵIndiana
Utah

Edinburgh
Glasgow

Liverpool
SwanseaSouthampton

ＦＮＡＬ

Berlin/Zeuthen
DESY/Neumann
Bielefeld
Regensburg

Future expansion to Asia/Oceania

BNL/Columbia

ＵＣＳＢ

St．Louise

JLAB

Arizona

Germany
LatFor
networkJapan
LFT Forum
network

KEK

Future expansion to EU network
Italy, France, Spain, Denmark,…

MIT/Boston U

京都大学

筑波大学

広島大学

3ILDG ProjectVideo Conference twice in a year (since 2002)


Two Working GroupILDG 11 will be held at the end of this November.

Metadata Working Group Æ Prof. Yoshie’s talkMiddleware Working Group

Our ContributionContribution to the WG (Meta Data, Middleware)
Main developer in Japan
Host Video conference of ILDG
We hosted face-to-face Middleware WG meeting in 2005.
Host ILDG interface from Japan to ILDG
Development of JLDG: national data Grid
Supported by NII CSI (National Institute for Informatics, Cyber
Science Infrastructure) Project4

ILDG Project: Middleware Working Group

ObjectivesMembers

Define the architecture and components (not development)
Define the interface of each components

Bálint Joó(Jlab, USA), G. Beckett (EPCC, UK) , D. Byrne (EPCC, UK), P.
Coddington (CSSM, Australia) , P. Dirk (DESY, Germany) , M.Sato(CCS, Convener) , Ishii (CCS) J. Simone (Fermilab, USA), W. Watson(Jlab, USA, Convener)

Chronology and Main activities

2003/01: WG organized
2004/10: Interface agreement at F2F Meeting at EPCC
Monthly Tel conference
F2F Meeting once in year
5

Review: Data Grid Web Services Architecture
Web ServicesMeta Data Catalog (MDC)
Replica Catalog (RC)Replication Service
ILDG Client

SRM Service

File Server(s)(Consistency Agent)

Storage (disk, silo)Single Site6

Grid-of-Grids
Clients can access multiple MDC’s at different sites
 Directory service tells the locations of MDC’s, which may be
managed by LDAP
userWSwebbrowserserver-sideclient, oruser app

WS

MDC

Japan

MDC

UK

MDC

USWS
MDC
directory
service7

Status Report: ILDG Middleware(at Last ILDG10)JLDG site: www.jldg.org is in production

MDC WS
 Using “Exist” XML database. We need to update with a new version.
RC WS
 Simple WS : one-to-one mapping from LFN to Transfer URL (http) We plan to replace with a new FC interface.
No SRM

File transfer

httpd for “wget” under simple access control by .htaccessgsiftp file transfer is also available. It checked by Interoperable testing .
 gridmap-file will be sync’ed automatically with ILDG VOMS (very soon!)

QCDArchive: Lattice QCD Archive, Japan

WS-enabled version of Server-side client

MDC WS interface is used for MetaData Search

No progress since the last ILDG98

Lattice QCD Archive: interface to ILDGWeb-service-enabled version of server-side client,
QCDArchive

See http://jldg-www.ccs.tsukuba.ac.jp/QCDArchive/
Two user web interfaces
Interactive ensemble search– specific to our configuration data
Raw Xpath Query interface
for Ensemble data and Configuration Data




MDC
WS
query

metadatatransfer
Configuration
データ

Download script is supposed
to be executed off-line.

Status

html
userserversideclient

Listing the matched data
Generating download scripts
webbrowser

Need some brush-up on design
No actual data is stored yet.

PlanAdd the function to switch to other site
search over multiple sites
9JLDG: Japan Lattice Data GridNew grid infrastructure for LQCD
community in Japan
File Sharing with “GFarm” Global FileSystem

GSI authentication managed by VOMS

http://datafarm.apgrid.org

JLDG:
JLDG:infrastructure
infrastructureof
ofQCD
QCD
collaboration
insideJapan
collaboration inside Japan

Use NII/NAREGI CA

Access to configurations from ILDG iscontrolled by registering with metadata

A part of data can be accessed fromILDG through www.jldg.org

KEKCCS, Tsukuba
Kanazawa universityNew domain: jldg.org
 A gateway to JLDG : www.jldg.org

Kyoto University

U. HIroshima

Data in each site is shared by“Gfarm” global file systemover the grid.
10Status: JLDG
Gfarm servers at 5 Sites have been setup

U. Tsukuba (CCS), KEK, Osaka U, Kyoto U, Hiroshima U.

Local JLDG CA/RA and VOMS are ready (running at Tsukuba)
Now under testing between sites.
KEK site
U. Tsukuba(CCS)

ILDG
internet

Osaka sitewww.jldg.org

Gfarmserver

Gfarm
MetaServer
Gfarm
serverTsukubasite

Gfarmserver

Kyoto SiteSuperSINETJLDG local
CA/RAVOMS

Gfarm
serverGfarm
serverHiroshima Site
11