Instructions for data analysis software:- Preparation- The IUGONET project and its products for space weather study- Installation- How to Use SPEDAS, part1- How to Use SPEDAS, part2
Published by IUGONET Project Team, Sep. 2017.
http://www.iugonet.org/?lang=en

The IUGONET project and hands onseminar for its data analysis software

Characteristics of Upper Atmosphere

1.
2.
3.

Affected by various phenomena from the earth surface to the space
Many physical parameters
Various variations including solar activity are overlappedGround Observations for Upper Atmosphere

JpGU 2014

Objectives of the IUGONET
Problem

SolutionIUGONET provides a new research platform thatenables metadata extracted from ground-based
observation data to be shared.
In addition, IUGONET developed analysis
software to access and analyze data in an
integrated fashion.

Observational data should be quality
controlled and managed by the specialists
who know the observations.
For users….It was not easy to reach anecessary information, since databases
are distributed in various universities andinstitutes.

Overview of the project
The Inter-university Upper atmosphere Global Observation
NETwork (IUGONET) project aims at establishing “einfrastructure” for researchers to effectively find, get, andanalyze various kinds of upper atmospheric data spreadover Japanese universities and institutes.
 To exchange ground-based observation data accumulated over
50 years since IGY (both digital and analogue data)

Planetary Plasma andAtmospheric ResearchCenter To promote analyses of multi-disciplinary data, which will lead tocomprehensive studies of mechanisms of long-term variations in
the upper atmosphereTohoku University

Kwasan and Hida
Observatories
WDS for GeomagnetismInternational Center forSpace Weather Scienceand Education

Kyoto University
WDS for Aurora

Data Analysis Center for
Geomagnetism and Space
Magnetism

National Institute ofPolar Research

Kyoto University

Kyushu University

Solar Terrestrial Environment
Laboratory
Research Institute for
Sustainable Humanosphere

Nagoya University

Kyoto University

WDS for Cosmic Rays

6©2011 Google - Map data ©2011 Geocentre Consulting, ZENRIN, Europa Technologies, Mapabc, SK M&C -Schematics of the projectInternational
Users

Expand the system to other
Geoscience community

Kyushu Univ.
ICSWSEMD

NagoyaUniv.
STELDevelopers
MD

WDC for Cosmic Rays

IUGONET
Information CenterMDMD

TohokuUniv.
PPARC

Kyoto Univ.

MDMD

NIPR
WDC for Aurora

Chairperson

MD
WDC forgeomagnetism
SpaceMagnetism

Kyoto Univ.
DACGSM

Kyoto Univ.
RISH

AstronomicalObservatories

IUGONET Metadata Format/Structure
IUGONET metadata format = SPASE + modifications(http://www.spase-group.org)

What’s SPASE?

metadata format developed by international
consortium to comprehensively describe
research resources regarding heliospheric andmagnetospheric satellite observations

 closely related to STP and upper atmosphereresearches ( easy to use as a base format)
 new metadata elements & words appendable (customizable according to our data ) widely-used in VxOs ( possible metadataexchange in the future)

IUGONET’s modifications
 additional words to represent non-digital archives
 additional words to represent heliospheric coordinates new metadata elements to describe observation location & range

IUGONET Metadata Database IUGONET MDB (called IUGONET Type-A) is capable of cross-searchingobservational data distributed across the IUGONET institutions.
 IUGONET Type-A brings a remarkable advancement in accessibility to theobservational data and accelerate the interdisciplinary study.
 IUGONET Type-A provides a one-stop web services such as
searching data, finding interesting events, interactively plotting the data,and leading users to more detailed analysis.

Analysis Software SPEDAS• The IUGONET Data Analysis Software(UDAS) is the plug-in software for Space
Physics Environment Data Analysis
System(SPEDAS), formerly known as
THEMIS Data Analysis Software suite (TDAS)• The IUGONET data (e.g., geomagnetic data,
aurora data, radar data, and so forth) andmany satellite mission data (THEMIS, GOES,WIND, and ACE) can be handled.

SPEDAS/TDAS• It is possible to use many routines tovisualize and analyze time series data.
• It accesses the IUGONET data through theInternet, and then the data areautomatically downloaded onto the user's
computer

Relationship between
UDAS, SPEDAS, and IDL

Outline of Loading/Plotting Data Using SPEDAS
Data Servers onthe InternetUser PC
Automaticdownload
data

Data areloaded as
tplot variables

SSL, Berkeley,
THEMIS, GBO
CDAWeb, OMNI,ACE, Wind, etc.

Directories for thedownloaded data arecreated automatically.

datadata

Data can be easily plotted, for example, by only three basis commands
with the SPEDAS-CUI tool.
1. Set a time period
2. Load *** data
3. Plot the loaded datatimespan, ‘yyyy-mm-dd’
iug_load_***tplot, +++If using the GUI tool, only a few simple clicks of your mouse are required to make thesame plot as that created by the above command with the CUI tool

Examples of SPEDAS visualization
Time series stack plot

Correlation

MappingWavelet

Contributions for Space weather study
Sun

Data supported by SPEDAS

MagnetosphereSatellite data
Stereo SOHOWindACEIMP-8THEMIS
Satellite

LANL

NASA
OMNI

ERG

GOES

VAP Satellite
EFW
RBSPICEGround-based observational data
THEMIS
Geomag.

THEMIS
CameraCARISMA
Geomag.

GIMAGeomag.

Greenland
Geomag.

MACCS
Geomag.

USGSGeomag.

Solar
Telescope

THEMIS Satellite

IUGONET, ERG
Solar Telescope, Solar and planetary radio
telescope, Ionosphere radar (SuperDARN,EISCAT, etc.), Atmosphere radar (MU, EAR,etc.), Meteorological observation data,Geomag. network（WDC, MAGDAS,
210MM, Antarctica・Iceland, etc.）

Atmosphere& Ionosphere

Interdisciplinary study
 Many missions have provided plugins for SPEDAS.
 IUGONET has also provided a plugin for SPEDAS, which includes manyroutines for loading various ground-based observation data.
 SPEDAS is suitable for Space Weather study.

Summary and Future Vision
NGDC
…

NSSDC
…

Big DataCenters

VMO
VHO

Middle Class DC

VEPO
VITMO

Virtual Obervatory(US STP community)

VSPO

ViRBO

(Project of EU STPcommunity)

Various Geosciences
Databases in JapanICSU - WDS
ICSU - CODATA(International Framework)

Japanese STP
community

IUGONET is a basis of a network
datacenter

IUGONET includes three
WDC members-> Functioning like a big data
center efficiently

・Kyoto Univ.(Geomagnetism)
In progress
・STEL(Cosmic Ray)・NIPR(Aurora)Event we analyze today
Geomagnetic storms followed by X-class flares and coronal massejections (CMEs) occurred on March 5-11, 2012.

GOES Ｘ-ray flux

X-class flare

From Space Weather News
http://swnews.jp/

Introduction of
IUGONET Type-A

How to use IUGONET Type-A
1. Access to IUGONET Type-A (http://search.iugonet.org/)

17

How to use IUGONET Type-A
Search Data

1. Set Timespan“2012/03/04 to 2012/03/10”

2. Click “Search” button

18

How to use IUGONET Type-A
Search Results

1. Click Plot
Search results are listed here.

19How to use IUGONET Type-A
Search Results

Search results are displayed with QL plots

You can select a timeinterval from 1, 3, or 7days for time-series data.
If you select 7-days, datafrom 2012/03/04 to2012/03/10 are displayed.

Images on the last
day of the selectedinterval are displayed(i.e., 2012/03/10 inthis case)
Scroll down
20How to use IUGONET Type-A
Search Results

Geomagnetic Indices

1. Click Dst index provisional

21

How to use IUGONET Type-A
Data Information(Metadata)

QL plot of Dst index

Set date（this is the last day of the interval）

To previous day

To next day

You can change
timespan here

Scroll down
22How to use IUGONET Type-A
Data Information (Metadata)
Acknowledgement
Contact Person

Access Information

Scroll down
23How to use IUGONET Type-A“How to plot” section shows how to plot the data by the dedicated software“IDL/SPEDAS”.
CUI #Basic： SPEDAS
commands minimally required
to plot the data
CUI #Advanced：
SPEDAS commands
used to create QL
plot in the metadata
display page.
GUI：How to plot the datawith SPEDAS-GUI.

You can plot Dst index with the SPEDAS-GUI tool according to this instruction.

24

How to use IUGONET Type-A
Interactive data plot (UDAS web)

Scroll up

Check the checkboxes.
You can select 3 data at most.
25

How to use IUGONET Type-A
Interactive data plot (UDAS web)

Click Create button
26

How to use IUGONET Type-A
Interactive data plot (UDAS web)

Select data that you want to plot,
then click “Plot”.

27

Installation of SPEDASPreparation of SPEDAS
Download of SPEDAS GUI tool
1. Access the THEMIS satellite software website
http://spedas.org/wiki/index.php?title=Downloads_and_Installation
2. Scroll down to “Download” section

Click the proper link for your OS.
The compressed executable file
will be downloaded in several
seconds or minutes.

Start of IDL-VM(GUI) tool
[1] Unzip the downloaded zip file.
[2] Double-click the executable file
named ‘spedas’ in the directory‘spedas_v_2/spd_gui’.

Doule-click the executable
file named ‘spedas’

Click the icon
‘spd_gui’.

[3] IDL Virtual Machine window opens on
your PC, so please click the ‘spd_gui’button.

Start of IDL-VM(GUI) tool

Ready?

How to Use SPEDAS
part1
・Load data
・Plot data
・Save figure, data, and your work

GUI Basic Operation
Lesson:Load Dst index
1. Click this icon

Or,Select “Load Data” from “File” dropdown menu.

GUI Basic Operation
1. Click IUGONET Tab

2. Uncheck “Use Single Day”

4. Chenge Instrument Typegeomagnetic_field_index

3. Set Date and Time
Start Time: 2012-03-04/00:00:00
Stop Time: 2012-03-11/00:00:00

GUI Basic Operation

2. Click the arrow
1. Choose three parameters- Dst_index- *(all)- *GUI Basic Operation

Display of Data Use Policy

1. Click“OK”

GUI Basic Operation

1. Data was loaded successfully!

2. Click“Done”

GUI Basic Operation
Lesson:Plot data
1. Click this icon

Or,
Select “Plot/Layout Options…” from “Graph”
dropdown menu.

GUI Basic Operation

2. Click “Line”

1. Select data which you want to plot:
wdc_mag_dst_prov

GUI Basic Operation

1. Selected variable nameis added to this box

2. Click OK

GUI Basic Operation
PanelX(Y,Z)
X: panel serial number
Y: row index of the panel
Z: column index of the panel

You can add, remove, and edit
panels with these buttons.
You can move panels
and change the numberof panels per page.

Lock to panel:
Lock panel axes tocurrently selected panel.

GUI Basic Operation
Try:
Load other three data
AE index(in IUGONET tab)
Instrument: magnetometer
MAGDAS magnetometer(in IUGONET tab)
Instrument: magnetometer
Station: HER
MAGDAS magnetometer(in IUGONET tab)
Instrument: magnetometer
Station: ASB

GUI Basic Operation
Answer

1. Select IUGONET tab

2. Select magnetic_field_index
4. Click arrow
3. Select AE_index, *(all), *5. Data is loaded

GUI Basic Operation
Answer
1. Select geomagnetic_field_fluxgate
3. Click arrow
4. Data is loaded
2. Select magdas#, asb & her, *5. Click DoneGUI Basic Operation
Lesson:Add Plot
wdc_mag_ae_prov_1min

1. Click this icon

Or,
Select “Plot/Layout Options…” from “Graph”
dropdown menu.

GUI Basic Operation
1. Click Add

3. Click “Line”
4. Data are added
2. Select wdc_mag_ae_prov_1min

5. Click OK

GUI Basic Operation
1. Click black triangles

GUI Basic Operation
Try:
Plot other two data
magdas_mag_her_1sec_hdz
magdas_mag_asb_1sec_hdz

GUI Basic Operation
Answer

2. Click Line

3. Data are added
1. Select magdas_mag_her_1sec_hdz

GUI Basic Operation
Answer

2. Click Line

1. Select magdas_mag_asb_1sec_hdz

4. Click OK

3. Data are added

GUI Basic Operation

GUI Basic Operation
Lesson:Save plot as figure
1. Select
File – Export To Image File

2. Select save folder

2. Input file name and select format (by extension)

3. Click “save”

GUI Basic Operation
Confirmation

Back to previous

Click “Save”

GUI Basic Operation

X pixel 856, Y pixel 554(65%)

X pixel 428, Y pixel 554

GUI Basic Operation
Lesson:Save data as ascii
1. Select
File – Save Data AsGUI Basic Operation
2. check this box
3. Select time interval
1. Select data which you want to savemagdas_mag_her_1sec_hdz_x
4. check this box

6. Select save folder

7. Input file name (data is saved in csv format)
5. Click Save
8. Click “save”

GUI Basic Operation
An ascii data file was
successfully saved!!!

GUI Basic Operation
Lesson:Dump your workspace
1. Select
File – Save SPEDAS Document

2. Select save folder

3. Input file name

4. Click “save”※ SPEDAS Document is written in XML format

How to Use SPEDAS
part2
・Restore your work
・Manage axis
・Process and data

GUI Basic Operation
Lesson:Restore part1 workspace

1. Exit SPEDAS
2. Run SPEDAS again
3. Select File- Open SPEDAS Document
4. Click “Yes”
5. Select the saved tgd file.

GUI Basic Operation
Lesson:Remove plot

1. Select
Graph – Plot/Layout Options

GUI Basic Operation
2. Click “Remove
1. Select
wdc_mag_ae_prov_1min_time -vs- wdc_mag_ae_prov_1min_4
in the right-hand panel.

3. Remove
wdc_mag_ae_prov_1min_time -vs- wdc_mag_ae_prov_1min_3
wdc_mag_ae_prov_1min_time -vs- wdc_mag_ae_prov_1min_2
wdc_mag_ae_prov_1min_time -vs- wdc_mag_ae_prov_1min_1
in the same way

4. Click OK

GUI Basic Operation
ResultGUI Basic Operation
Lesson:Change X range (time scale) ofthe plot (1)Reduces X range bymajor tick marker
Expands X range bymajor tick marker
Shift left X range bymajor tick marker
Shift right X range
by major tick marker

Use these icons

GUI Basic Operation
Lesson:Change X range (time scale) of the plot (2)1. Select
Graph – X Axis Options

GUI Basic Operation
1. Select (L) Panel 1 (1, 1) 2. Select Fixed Range

3. Change values
Min 2012-03-06/00:00:00.000
Max 2012-03-11/00:00:00.000

4. Click OK

GUI Basic Operation
Lesson:Customize Ticks.
1. Select Major Ticks By
Number or Interval.
2. Input the number into
# (Number or Interval)of Minor Ticks
3. Click "Apply to All Panels“,
then click "OK“.

GUI Basic Operation
Lesson:Change Annotations

1. Select your favorite
format in the pull-downmenu of Annotation Format.

2. If you want tochange the character
font, size, and color,select your favorite
format in the pull-downmenu here.

3. Click "Apply to All Panels“,
then click "OK“.

68GUI Basic Operation
Lesson:Customize Labels (of X axis)

1. Select the bottom panel
number (“Panel 4” )2. Check the“Show Label”
box

3. Type “Universal Time ”
on the Edit/Add Label
4. Click "OK“
Note: Never use "Apply to All Panels“GUI Basic Operation
ResultGUI Basic Operation
Other options.
• Page Options…
Customize the text and layout of thepage.
• Panel Options…
Customize the title and color of eachpanel.
• Line Options…
Customize the line and symbol of eachplot panel.
• Legend Options…
Customize the legend which appears
when you put the mouse cursor on theplot.
• Variable Options…
Display the values of the selectedparameters under the time label.

GUI Basic Operation
Lesson:Reset X range (time scale)
1. Select X Axis Options

GUI Basic Operation
1. Select (L) Panel 1(1, 1) 2. Select Auto Range

3. Click “OK”

GUI Basic Operation
Lesson:Processing- subtract average

1. SelectAnalysis – Data Processing

GUI Basic Operation
4. Click Subtract Average

3. Active Data are added
2. Click right arrow

1. Select data you want to process
magdas_mag_asb_1sec_hdz
magdas_mag_her_1sec_hdz

GUI Basic Operation

New variables are created.
magdas_mag_asb_1sec_hdz-d
magdas_mag_her_1sec_hdz-d

1. Click DoneGUI Basic Operation
Open “Plot/Layout Options”

1. Remove Panel 3 and 4

3. Click line
4. Data are added

2. Selectmagdas_mag_her_1sec_hdz-d

5. Then, add the other variable,
magdas_mag_asb_1sec_hdz-d
to panel in the same way.

6. Click OK

GUI Basic Operation

Subtracted average!

GUI Basic Operation
Try:
Plot Power Spectrum of
magdas_mag_her_1sec_hdz_x

Hint1: Use “Data Processing” forcalculating (if you will get an
option dialog, use default value)
Hint2: Use “Spec” for plotting

1. SelectAnalysis – Data Processing

GUI Basic Operation

2. Active Data is removed

1. Click Clear Active

GUI Basic Operation

3. Active Data are added
2. Click right arrow

1. Select data
magdas_mag_her_1sec_hdz

5. Click OK

4. Click Power Spectrum

GUI Basic Operation

1. New variables are created!

2. Click DoneGUI Basic Operation
Open “Plot/Layout Options”

2. Click Spec
3. Data are added
1. Selectmagdas_mag_her_1sec_hdz_x_dpwrspc

4. Click OKGUI Basic Operation
ResultThis color bar can becustomized inGraph - Z Axis Options.

GUI Basic Operation
Lesson:Use Calculate (Equation editor)
1. SelectAnalysis – Calculate…

GUI Basic Operation
Equation Editor for SPEDAS
Variable=Your loaded data

Built-in function

Built-in Operator

Built-in Constant

GUI Basic Operation
Lesson:Make an equation using the loaded variables.

1. Type variable/function/Operator/Constant directly

Note:
Enclose the variable in double quotation marks

GUI Basic Operation
Lesson:Make an equation using the loaded variables.
1. Select magdas_mag_her_1sec_hdz-d_x
3. Variable is added
2. Click arrow

Then, try to add the offset (+200) to
magdas_mag_her_1sec_hdz-d_x

GUI Basic Operation
Note: one line, never return

Answer"magdas_mag_her_1sec_hdz-d_x_ofst" = "magdas_mag_her_1sec_hdz-d_x" + 200

1. Click Run
2. A new variable is created
3. Click done

GUI Basic Operation
Open “Plot/Layout Options”

1. Select Panel 3 and Remove it

3. Click Line

4. Data are added

2. Selectmagdas_mag_her_1sec_hdz-d_y
magdas_mag_her_1sec_hdz-d_x_ofst

GUI Basic Operation
Open “Plot/Layout Options”

1. Change the value of Row to 3

2. Panel 6 is changed to (3, 1)

3. Click OK

GUI Basic Operation
ResultOffset (+200nT) wasadded to the H comp.

GUI Basic Operation
Try:
Expand the plot using the mouse.
By left-click anddrag the mouse

A new page opens

GUI Basic Operation
Lesson:Load data from CDAWeb.

1. SelectFile – Load Data using CDAWebGUI Basic Operation
2. Select Instrument Type
Magnetic fields (space)....

Open “CDAWeb Data Chooser”

1. Select Mission
OMNI(Combined …)
3. Click ”Find Datasets”

4. Data are listed

GUI Basic Operation

1. Selected data thatyou want to load

2. Set Date & Time

3. Click “Get CDAWeb Data”

GUI Basic Operation
Open some pop-up windows to confirm the data

1. Click “OK”

3. Finally, click “Close” in the“CDAWeb Data Chooser” window
2. Click “OK”

GUI Basic Operation
Open “Plot/Layout Options”

2. Click Line
1. Select
the listed all data in term

3. Data are added

4. Click OK

GUI Basic Operation
ResultAll solar windparameters are plotted.

For advance…
UDAS website：http://www.iugonet.org/product/analysis.jsp

Acknowledgment
SPEDAS is a grass-roots data analysis software for the Space Physics community, which wasdeveloped by scientists and programmers of the UC Berkeley's Space Sciences Laboratory,
UCLA's IGPP and other contributors

http://www.iugonet.org/