# Presentation: 424264

## ICALEPCS 2007, Knoxville 15-19/10/2007 – ra12-1Data Management at JET with a look forward to ITER

**ICALEPCS 2007, Knoxville 15-19/10/2007 **– ra12-1**Data Management at JET with a look forward to ITER**

- J.W. Farthing(1) (jonathan.farthing@jet.uk)
**CO**ntrol and **D**ata **A**cquisition **S**ystems (**CODAS**), Machine Operations, UKAEA Culham Division, Culham Science Centre
- T.Budd(1), A.J.Capel(1), N.Cook(1), A.M.Edwards(1), R.Felton(1), F.S.Griph(1), E.M.Jones(1), R.A.Layne(1), P.A.McCullen(1), D.McDonald(1), M.R.Wheatley(1) and JET-EFDA Contributors[*],
- M.Greenwald(2), J.B.Lister(3) and J.Vega(4)
*1) EURATOM/UKAEA Fusion Association, Culham Science Centre, Abingdon, OX14 3DB, UK*
*2) **Plasma Science and Fusion Center, MIT, Massachussets, USA*
*3) CRPP-EPFL, Lausanne, Switzerland*
*4) Asociacion EURATOM/CIEMAT para Fusion, Spain*
- [*] See Annex 1 of M.L.Watkins, “*Overview of JET Results*”, OV-1.3, Proceedings 21st IAEA Fusion Energy Conference, Chengdu, China, 16-22 October 2006.

## JET (Joint European Torus) Largest Tokamak to date Day 1, June 1983

**JET**** ****(Joint European Torus) ****Largest Tokamak to date**** Day 1, June 1983**

## JET (www.jet.efda.org)    ITER (www.iter.org)

**JET**** (****www.jet.efda.org****)    ****ITER**** (**_**[www.iter.org](http://www.iter.org/)**_**)**

- JET Enhancements
- ITER relevant Engineering and Physics

- 2x2x2
- Plasma Volume x10

## Data Management at JET

**Not all as integrated as we would like**

**JET Control Systems**
**Experimental setup**
** ****Control**
**Data Acquisition**
**Data Analysis**
**Data Storage**

## Hierarchical and Modular Architecture

**Hierarchical and Modular Architecture**

- Three-level hierarchical and modular control system structure in hardware and software:
- Central/Supervisory 	- level 1
- Subsystem	  	- level 2
- Component 	      	- level 3

- 1

- 2

- 3

## JET Experiment Pulse Cycle

**JET Experiment Pulse Cycle**

**JET is pulsed**
- 25 pulses per day
- ~70,000 since 1983
- Two shifts 06:30 – 22:30
- Pulse every ~ 30 minutes
- ~30-40s of plasma
- Maximise Repetition Rate

**ITER will be pseudo-continuous**
**(60mins of plasma)**

## Pulse Preparation, Setup, Validation

## Control System Databases

**Control System Databases**

## Real-time Control

- Ongoing growth area on JET
- Fundamental on ITER
- Streaming data
- Real-time analysis, control, protection
- See Rob Felton poster – WPPA22

## Data Collection,Transfer,Storage,Analysis

**Data Collection,Transfer,Storage,Analysis**

## Intershot Data Analysis

- ~80 codes are run intershot
- Database models dependencies of codes on raw data and each other (processed data)
- Critical path analysis
- Codes scheduled in parallel based on dependencies
- Multi-tier collection strategy
- Fine tuning of raw data availability
- Critical part of intershot is complete before end of data collection
- Communication via processed data files
- Some work on web using services
- Integrated data analysis
- JET-like intershot analysis will not exist on ITER
- Must be available in real-time

**Intershot Data Analysis**

## Mass Data Store

- Raw Data (IPF,QPF,JPF,DPF,LPF) – Read-Only
- All data accessed via subsystem + pulse number(=file) and signal name
- Home-grown (1981) data format, File headers contain signal information
- Client-server data access
- Home-grown file system-based indexing system holding meta data, ZFS
- Processed Data (PPF) – Read / Write
- Client-server data access
- MIMER RDBMS
- Meta data indexing system
- Access by pulse number, signal name
- NetCDF data files
- File management same as for raw data
- Central Physics File (CPF)
- SAS Database
- Access via SAS or SQL
- Migrating to Postgres for the underlying data
- MDSplus provides remote data access

## See www.mdsplus.org
MIT, CNR RFX-Padova, LANL – first developed late-1987
“.. allows all data from an experiment or simulation code to be stored into a single, self-descriptive, hierarchical structure” 
JET provides MDSplus glue layer to raw (JPF) and processed (PPF) data
Most Tokamaks provide MDSplus server
Many analysis and display programs are MDSplus aware
Installed in > 30 Labs world-wide, De-facto standard (ITER will keep concept)
Provides Remote Data Access (RDA)
Globus version using X.509 certificates for authentication, e.g. remote writes

- MIT, CNR RFX-Padova, LANL – first developed late-1987
- “.. allows all data from an experiment or simulation code to be stored into a single, self-descriptive, hierarchical structure”
- JET provides MDSplus glue layer to raw (JPF) and processed (PPF) data
- Most Tokamaks provide MDSplus server
- Many analysis and display programs are MDSplus aware
- Installed in > 30 Labs world-wide, De-facto standard (ITER will keep concept)
- Provides Remote Data Access (RDA)
- Globus version using X.509 certificates for authentication, e.g. remote writes

**MDSplus – Model Data System**

- MDSplus provides the basic **functional** requirements of the ITER data access system from the users perspective
- Extensions have been proposed to cover continuous data acquisition as part of the ITER conceptual design

## JET Experimental Data Volumes

- ~10GB / pulse = 0.25TB/day
- 2010 ~60GB / pulse ?
- c.f. LHC 10PB/year ?

- 2000

- Processed data <= 10% of Total

- Simulation data volumes are becoming significant

- ~40TB in 25 years

## Where will ITER be different? (1)

- Long-pulse / Pseudo-continuous
- Merge pulsed and continuous data acquisition
- Streaming data with varying sampling rates
- Same for all data sources e.g. CODAC internal data
- Single access layer to all data
- Single signal naming scheme as part of a wider plant naming scheme
- Rules to be developed, URIs (RFC 3986), managed namespaces, Qnames
- Data access via absolute time
- But pulse number and relative times to events in pulses / segments
- Novel data mining and data classification techniques

- See Jo Lister talk “Status of the ITER-CODAC (_**CO**_ntrol _**D**_ata _**A**_ccess and _**C**_ommunication) Conceptual Design”, ID=1422, 08:30, Thursday

## All fusion data is time series

**All fusion data is time series**

- Long-pulse experiments imply new techniques for data analysis

## Pattern recognition – RDBMS + SQL

- Encoding

- Data encoded according to discrete set of values (code alphabet) f(gradient)
- Pattern recognition via string comparison
**Alternative Methods**
- Pattern recognition can be based on concavities or other features of the signals

- J.Vega CIEMAT

## Where will ITER be different? (2)

- All project data can have an associated time
- Requirement to play back history – training or enquiries into incidents
- Software, configuration data as well as plant data
- Concept of future as well as past e.g. for experiment planning purposes
- Full data provenance
- publications back to ADC boards, firmware, software versions ...

- Slow control (e.g. machine conditioning) = Plasma Control
- Single Schedule Editor and real-time Scheduler is required?
- Just a question of scheduling with different time scales?
- Extend to Experiment planning, Maintenance scheduling, Shutdown planning?
- Campaign = Year, Shutdown planning=year/month/day, Experiment = Week/Day, Session = Day/Half-Day, Pulse=Hour, Machine conditioning=Hour, Maintenance scheduling=Month, Segment in pulse=Minute, Control mode=Second, Plasma Event=millisecond

- Real-time analysis, control, protection
- General trend on fusion devices today, See Rob Felton poster – WPPA22

## Where will ITER be different? (3)

- Data Volumes and Rates
- ~10PB/year in 2016? c.f. LHC
- ~10GB/s
- Underestimates?
- 1977 JET design figures JET suggested that less than 20GB data would be collected to the end of the project life in 1990. (170GB were collected).
- Much more modelling, model data set sizes will be significant
- Simulations running in real-time, comparable data volumes?
- 1m of burning plasma simulation = 1TB
- Archive all raw data ? Only novel data ? ...
- Cost is small by comparison to ITER investment so archive all
- Data mirroring at partner sites?
- provides backup functionality

- Internet age
- Commercial / open-source solutions widely available
- Very little should be developed in-house

## ITER – Single Project-wide Database?

- Single logical database to last the project lifetime
- Documents, CAD, Structured Data, Plant Naming System, Signal Naming Scheme, Software, Planning, IT
- Construction, Operations, Maintenance, Decommissioning
- Administration, Engineering, Physics
- Experimental Planning, Operations, Data Storage, Analysis, Publications
- Control and Data Acquisition Systems
- Data driven and generated from the database
- All data has an associated time – keep history - full provenance
- Management Systems
- Processes, Quality, Risk Management, Obsolescence Management ...

- Self-description of the plant systems delivered “in-kind” – 7 partners
- Cubicles, modules, wiring, signals, software, firmware, history ...

## Summary

- JET hierarchical and modular system architecture
- Hierarchical ~1981 subsystem (level-2) databases still in use
- Level-1 control layered above - JET-wide setup, increased functionality
- Raw data pulse file data formats unchanged since ~1981, processed data file formats have changed
- Client-server - underlying technology changes hidden from end-users

- ITER is starting with a clean sheet
- All-embracing project-wide database to last the project lifetime
- Derivation of control system data

- JET has large number of heterogeneous support databases
- Developed at different times, by different groups and using different technologies – in-house and commercial systems
- Integration between them is not always ideal

- End

- backup slides

## ITER (2) – courtesy jo.lister@iter.org

## SQL:2006

- Merge RDBMS and XML technologies
- ”ISO/IEC 9075-14:2006 defines ways in which SQL can be used in conjunction with XML. It defines ways of importing and storing XML data in an SQL database, manipulating it within the database and publishing both XML and conventional SQL-data in XML form. In addition, it provides facilities that permit applications to integrate into their SQL code the use of XQuery, the XML Query Language published by the World Wide Web Consortium (W3C), to concurrently access ordinary SQL-data and XML documents” http://en.wikipedia.org/wiki/SQL

- Data-driven control system based on web services?

## ITER (4) – Data Volumes

**Simple Extrapolation from JET to ITER**
**JET 50-60GB/pulse 2010, 150 days, 25 pulses/day**
**ITER – Same number of plasma seconds/year**
**60 x 2 x 2 x 2 = 480GB/pulse in 2016**
**480 x 25 = 12TB/day x 150 = <10PB /year in 2016**
**c.f. LHC 10PB/year in 2007**

## ITER (3) – courtesy jo.lister@iter.org

## Mass Data Store Technology (1)

- SUN E4900 server
- 2 x (4 UltraSparc4 1.2GHz dual core processors with 32GB RAM) S10
- Zones used to isolate services
- Trunked 4*1Gb/s ethernet for incoming and outgoing data
- 3 separate disk areas
- Spool - 0.7TB ANS ATA-based mirrored disk for incoming data
- Cache - 60TB ANS ATA-based RAID cache disk – 57% full
- Raw data is compressed (ADPCM)
- Cost-based deletion algorithm
- Recalls – 0.5TB ANS ATA-based RAID
- ZFS
- Migrated from SUN Volume Manager and UFS during 2007
- Veritas Netbackup
- Not ideal. Replace by what ? Home-grown system? SAMFS ?
- 9940B (200GB native) StorageTek drives
- ACSLS tape management (Oracle)
- 3 tape copies - security

## Data File Management

- Cost-based deletion algorithm f(last access time,size) to manage the cache space

## JET provides key contributions to predict ITER performance

**JET provides key contributions to predict ITER performance**

******E**** ******** I****p**** R****2**

**Confinement time** ******E**** = W****plasma****/P****input**

## Data Access

- The data shows file access times although fusion data is analysed at signal level, not files
- Client-server technology, JPF server, PPF server, MDSplus server
- The servers retrieve files from tape and extract signals

## JET Prospects

- JET Extension through Framework Programme 7 in Support of ITER
- Shutdown March 2007
- ITER-like RF Antenna
- High Frequency Pellet Injector
- Many diagnostics
- November 2008
- Plasma Control Upgrade
- June 2009
- ITER-like Wall (Be)
- ITER-like Divertor (W)
- Neutral Beam Power Increase
- Many diagnostics
- Extra 43GB / pulse in 2010

- Modelling storage requirements
- 1m of ITER integrated burning plasma simulation yields 1TB data?

## Electra Database

## Module and Functional Position Histories

- Expanded FP History. Follow history of FP and installed modules at any level in the hierarchy

- Unusual to have such a detailed audit-trail

## Stock Management

- Min, Max number of spares to be held

- Spares are different from Stores
- Automatic Shortage Reports

## Object Monitoring System (OMS)

**plant://pf/tstatus:1 **specifies a live-database signal
**alarm://mc/alarm-ident **specifies an alarm
**alarm://mc/alarm-ident?help **specifies the *help text *for an alarm
**level1:/l1/parameter **specifies a Level 1 parameter from the default database
**level1://ye/l1/parameter **specifies a Level 1 parameter from the YE database
**wave://tf/wave-form-name **specifies the latest version of a waveform
**wave://tf/wave-form-name:version **specifies the a particular version of a waveform
**udp://sa/ident **specifies a *User Defined Point*
**level1-array://tf/array-name **specifies an array
**real-time://pf/point **specifies a point from the *Real Time Acquisition *system
- (for use by applications such as xrtdisp).
**file://TF/jet/pf/etc/file **Specifies the file /jet/pf/etc/file should
- be monitored by the OMS server on TF
**cfr:number **specifies the text of a CODAS Fault Report
**jetlog:thing **specifies an item in the JET log.
**snmp:/command?target **specifies an SNMP command to be executed on a particular
- target
**xdc:/ **Specifies a complete list of ALL xterminal users
**xdc:/mmi-cn1-1 **specifies the xterminal users of terminals being managed
- by mmi-cn1-1
**ppf:/ppf-ident-string **specifies data from a PPF dataset on the IBM mainframe.
- It could be a list of signals in the PPF or the
- data associated with a particular signa

## Pulse Preparation – Level-1 Software

- Pulse Schedule Editor

- PLANT

- Previous
- Settings

- ~40,000 Previous Pulses

- Centralised
- Supervisory
- Control
- Database

- ~20,000 Parameters
- ~600 plug-in codes

- Pulse
- Schedules

- Organised by Task Force

**Config**

**Config**

**Config**

**(Remote)Session Leader**

**Engineer in Charge**

**Real-Time Control Expert **

**Pulse**
**Types**

**Pulse**
**Type**

**LOAD**

**EDIT**

**REF**

- Offline Flight Simulation

## JET Data Analysis

## Some of the other JET Databases

- Jet Logging
- Experiment comments, Fault Logging
- Statistics
- CPU, Network measurements etc, Capacity planning
- Experimental planning
- Machine configuration
- CODAS Support Databases
- Cubicle Design/Documentation, Inventory Management, Document Management
- Product handler – software releases and installations
- Diagnostic Data Handbook
- Explains the meanings of the measurements
- Solaris Control System
- NIS
- Access control database – user, role of user, location of user, state of JET
- IT Systems
- Account management, Management Systems, Hardware, Software requests, Planning, Maintenance scheduling
- Publications / Pinboard

- Not all as integrated as we would like

## “Patterns in signals” approach: time-series data. An example in JET

- ECE signals: electron temperature

- All patterns follow the same behaviour but during different time
-  A fall
-  A growing slope
-  More abrupt in the 3rd case
-  A flat zone
-  A fast rise
-  A flat top

## Plant Systems delivered “in-kind”

- Self-description of the plant systems delivered “in-kind”
- Cubicles, modules, wiring, signals, software, firmware, history ...
- Merged into centralised CODAC / ITER-wide database
- Schemas supplied by ITER
- SQL 2006 Standard - ISO/IEC 9075-14:2006 – Merges SQL and XML
- Single plant and signal naming scheme
- Rules to be developed
- Universal Resource Identifiers (URIs). See RFC 3986
- http://www.iter.org/ns/   ns=namespace
- Qualified Names or Qnames
- e.g. signal: = http://www.iter.org/ns/signal/
- signal:neutrons/14mev_neutron_flux
- URL equivalent of URI gives meta data access
- Access to data as appropriate