Instructions for data analysis software:- Preparation- The IUGONET project and its products for space weather study- Installation- How to Use SPEDAS, part1- How to Use SPEDAS, part2
Published by IUGONET Project Team, Sep. 2017.
http://www.iugonet.org/?lang=en

Outline

This hands-on have 4 topics
1.
2.

Introduction to the IUGONET project
Data Analysis by using SPEDAS GUI toolbreak
3. Data Analysis by using web tool (IUGONET
Type-A) and SPEDAS CUI tool.
4. Application of IUGONET tools to analysis ofSyowa and Iceland data.

Characteristics of Upper Atmosphere

1.
2.
3.

Affected by various phenomena from the earth surface to the space
Many physical parameters
Various variations including solar activity are overlappedObjectives of the IUGONET
ProblemSolution
IUGONET provides a new research platformthat enables metadata extracted fromground-based observation data to be shared.
In addition, IUGONET developed analysis
software to access and analyze data in an
integrated fashion.

Observational data should be quality
controlled and managed by thespecialists who know the observations.
For users….It was not easy to reacha necessary information, sincedatabases are distributed in various
universities and institutes.

Overview of the project
The Inter-university Upper atmosphere Global Observation
NETwork (IUGONET) project aims at establishing “einfrastructure” for researchers to effectively find, get, andanalyze various kinds of upper atmospheric data spreadover Japanese universities and institutes.
 To exchange ground-based observation data accumulated over
50 years since IGY (both digital and analogue data)

Planetary Plasma andAtmospheric ResearchCenter To promote analyses of multi-disciplinary data, which will lead tocomprehensive studies of mechanisms of long-term variations inthe upper atmosphere
WDS for Geomagnetism
International Centerfor Space WeatherScience andEducation

Tohoku University

Kwasan and Hida
Observatories

Kyoto University
WDS for Aurora

Data Analysis Center
for Geomagnetism andSpace Magnetism

National Institute ofPolar Research

Kyoto University

Kyushu University

Solar Terrestrial
Environment Laboratory
Research Institute for
Sustainable HumanosphereKyoto University

Nagoya University
WDS for Cosmic Rays5©2011 Google - Map data ©2011 Geocentre Consulting, ZENRIN, Europa Technologies, Map

Ground Observations for Upper Atmosphere

JpGU 2014

Schematics of the projectInternational
Users

Expand the system tootherGeoscience community

Kyushu Univ.
ICSWSEMD

NagoyaUniv.
ISEEDevelopers
MD

WDC for Cosmic Rays

IUGONET
Information Center

MDMD

TohokuUniv.
PPARC

Kyoto Univ.

MD
MD

NIPR
WDC for Aurora

Chairperson

MD
WDC forgeomagnetism Space
Magnetism

Kyoto Univ.
DACGSM

Kyoto Univ.
RISH

Astronomical
Observatories

IUGONET Metadata Format/Structure
IUGONET metadata format = SPASE + modifications(http://www.spasegroup.org)

What’s SPASE?

metadata format developed byinternational consortium tocomprehensively describe researchresources regarding heliospheric andmagnetospheric satellite observations
 closely related to STP and upper atmosphereresearches ( easy to use as a base format)
 new metadata elements & words appendable( customizable according to our data ) widely-used in VxOs ( possible metadata
exchange in the future)

IUGONET’s modifications
 additional words to represent non-digital archives
 additional words to represent heliospheric coordinates new metadata elements to describe observation location & range

IUGONET Metadata Database

http://search.iugonet.org/
 IUGONET MDB (called IUGONET Type-A) is capable of cross-searchingobservational data distributed across the IUGONET institutions.
 IUGONET Type-A brings a remarkable advancement in accessibility to theobservational data and accelerate the interdisciplinary study.
 IUGONET Type-A provides a one-stop web services such as
searching data, finding interesting events, interactively plotting the data,
and leading users to more detailed analysis.

Analysis Software : SPEDAS• The IUGONET Data Analysis Software(UDAS) is the plug-in software for Space
Physics Environment Data Analysis
System(SPEDAS), formerly known as
THEMIS Data Analysis Software suite (TDAS)• The IUGONET data (e.g., geomagnetic data,aurora data, radar data, and so forth) andmany satellite mission data (THEMIS, GOES,WIND, and ACE) can be handled.

SPEDAS/TDAS• It is possible to use many routines tovisualize and analyze time series data.
• It accesses the IUGONET data through theInternet, and then the data are automaticallydownloaded onto the user's computer

Relationshipbetween UDAS,SPEDAS, and IDL

Outline of Loading/Plotting Data Using SPEDAS
Data Servers
on the InternetUserPC

Automaticdownload
data

Data areloaded as
tplot variables

SSL, Berkeley,THEMIS, GBO
CDAWeb, OMNI,ACE, Wind, etc.

Directories for thedownloaded data
are createdautomatically.

datadata

Data can be easily plotted, for example, by only three basis
commands with the SPEDAS-CUI tool.
1. Set a time period
2. Load *** data
3. Plot the loaded datatimespan, ‘yyyy-mm-dd’
iug_load_***tplot, +++If using the GUI tool, only a few simple clicks of your mouse are required to makethe same plot as that created by the above command with the CUI tool

Examples of SPEDAS visualization
Time series stack plot

Correlation

Mapping

WaveletContributions for Space weather study
Data supported by SPEDASSun

Magnetosphere

Satellite data
Stereo SOHO

Wind

ACE

IMP-8THEMIS
Satellite

LANL

NASA
OMNI

ERG

GOES

VAP Satellite
EFW
RBSPICEGround-based observational data
THEMIS
Geomag.

THEMIS
CameraCARISMA
Geomag.

GIMA
Geomag.

Greenland
Geomag.

MACCS
Geomag.

USGS
Geomag.

Solar
Telescope

THEMIS Satellite

IUGONET, ERG

Solar Telescope, Solar and planetary
radio telescope, Ionosphere radar(SuperDARN, EISCAT, etc.),
Atmosphere radar (MU, EAR, etc.),
Meteorological observation data,Geomag. network（WDC, MAGDAS,210MM, Antarctica・Iceland, etc.）

Atmosphere& Ionosphere

Interdisciplinary study
 Many missions have provided plugins for SPEDAS.
 IUGONET has also provided a plugin for SPEDAS, which includes manyroutines for loading various ground-based observation data.
 SPEDAS is suitable for Space Weather study.

Summary and Future Vision
NGDC
…

NSSDC
…

Big
DataCentersVHOMiddle Class DC

VMO VEPO
VITMO

Virtual Obervatory(US STP community)

VSPO

ViRBO(Project of EU STPcommunity)

ICSU - WDS
ICSU - CODATA(InternationalFramework)

VariousGeosciences
Databases inJapan

Japanese STPcommunity

IUGONET is a basis of anetwork datacenter

IUGONET includes
three WDC members-> Functioning like a big
data center efficiently

・Kyoto Univ.(Geomagnetism)
In・STEL(Cosmic Ray)・NIPR(Aurora)
progress

Hand on of SPEDASPreparation for hands on
Prepare 64 bit Operating System.
1. Access the following URL
http://themis.ssl.berkeley.edu/software.shtml
2. Download SPEDAS 3.1 zip file for your operating system(Win orMac), and then unzip it to your desktop.

Start of IDL-VM(GUI) tool
[1] Unzip the zipped SPEDAS file.
[2] Double-click the executable file
named ‘spedas’ in thedirectory‘spedas_v_3/spd_gui’.

Doule-click theexecutable file named‘spedas’

Click the icon
‘spd_gui’.

[3] IDL Virtual Machine window opens
on your PC, so please click the‘spd_gui’ button.

For Windows 10 users
If you encountered any graphics
problem on Windows 10, try launching
IDL in "Windows 7 Compatibility Mode".
To do that:
0. Right click on the IDL or SPEDAS
executable file and select "Properties".
1. Select "Compatibility" tab
2. Check "Run the program incompatibility mode for:"3. Select "Windows 7".
Click ok the apply the changes.
Restart the program and then try yourplot over again.

18

Start IDL-VM(GUI) tool

Ready?

How to Use SPEDAS
part1
・Load data
・Plot data
・Save figure, data, and your work

GUI Basic Operation
Lesson:Load Dst index
1. Click this icon

Or,
Select “Load Data from Plug-ins…” from“Data” dropdown menu.

GUI Basic Operation
1. Click IUGONET Tab

2. Uncheck “Use Single Day”

4. Chenge Instrument Typegeomagnetic_field_index

3. Set Date and Time
Start Time: 2012-03-04/00:00:00
Stop Time: 2012-03-11/00:00:00

GUI Basic Operation

2. Click the arrow
1. Choose three parameters- Dst_index- *(all)- *GUI Basic Operation

Display of Data Use Policy

1. Click“OK”

GUI Basic Operation

1. Data was loaded successfully!

2. Click“Done”

GUI Basic Operation
Lesson:Plot data
1. Click this icon

Or,
Select “Plot/Layout Options…” from “Plot”
dropdown menu.

GUI Basic Operation

2. Click “Line”

1. Select data which you want to plot:
wdc_mag_dst_prov

GUI Basic Operation

1. Selected variablename is added to thisbox

2. Click OK

GUI Basic Operation
PanelX(Y,Z)
X: panel serial number
Y: row index of the panel
Z: column index of the panel

You can add, remove, andedit panels with thesebuttons.
You can move panels
and change thenumber of panels perpage.

Lock to panel:
Lock panel axes tocurrently selected panel.

GUI Basic Operation
Try:
Load other threedata
AE index(in IUGONET tab)
Instrument:magnetometer
MAGDAS magnetometer(in IUGONET tab)
Instrument: magnetometer
Station: HERMAGDAS magnetometer(in IUGONET tab)
Instrument: magnetometer
Station: ASB

GUI Basic Operation
Answer

1. Select IUGONET tab

2. Select magnetic_field_index
4. Click arrow
3. Select AE_index, *(all), *5. Data is loaded

GUI Basic Operation
Answer
1. Select geomagnetic_field_fluxgate
3. Click arrow
4. Data is loaded
2. Select magdas#, asb & her, *5. Click DoneGUI Basic Operation
Lesson:Add Plot
wdc_mag_ae_prov_1min

1. Click this icon

Or,Select “Plot/Layout Options…” from “Plot”
dropdown menu.

GUI Basic Operation
1. Click Add

3. Click “Line”
4. Data are added
2. Select wdc_mag_ae_prov_1min

5. Click OK

GUI Basic Operation
1. Click black triangles

GUI Basic Operation
Try:
Plot other two data
magdas_mag_her_1sec_hdz
magdas_mag_asb_1sec_hdz

GUI Basic Operation
Answer

2. Click Line

3. Data are added
1. Select magdas_mag_her_1sec_hdz

GUI Basic Operation
Answer

2. Click Line

1. Select magdas_mag_asb_1sec_hdz

4. Click OK

3. Data are added

GUI Basic Operation

GUI Basic Operation
Lesson:Save plot as figure
1. Select
File – Save Page As Image File…

2. Select save folder

2. Input file name and select format (by extension)

3. Click “save”

GUI Basic Operation
Confirmation

Back to previous

Click “Save”

GUI Basic Operation

X pixel 856, Y pixel 554(65%)

X pixel 428, Y pixel 554

GUI Basic Operation
Lesson:Save data as ascii
1. Select
Data – Save Data As

GUI Basic Operation
2. check this box
3. Select time interval
1. Select data which you want to savemagdas_mag_her_1sec_hdz_x
4. check this box

6. Select save folder

7. Input file name (data is saved in csv format)
5. Click Save
8. Click “save”

GUI Basic Operation
An ascii data file was
successfully saved!!!

GUI Basic Operation
Lesson:Dump your workspace
1. Select
File – Save SPEDAS Document

2. Select save folder

3. Input file name

4. Click “save”※ SPEDAS Document is written in XML format

Coffee Break…

How to Use SPEDAS
part2
・Restore your work
・Manage axis
・Process and data

GUI Basic Operation
Lesson:Restore part1 workspace

1. Exit SPEDAS
2. Run SPEDAS again
3. Select File- Open SPEDAS Document
4. Click “Yes”
5. Select the saved tgd file.

GUI Basic Operation
Lesson:Remove plot

1. Select
Plot – Plot/Layout Options

GUI Basic Operation
2. Click “Remove
1. Select
wdc_mag_ae_prov_1min_time -vs- wdc_mag_ae_prov_1min_4
in the right-hand panel.

3. Remove
wdc_mag_ae_prov_1min_time -vs- wdc_mag_ae_prov_1min_3
wdc_mag_ae_prov_1min_time -vs- wdc_mag_ae_prov_1min_2
wdc_mag_ae_prov_1min_time -vs- wdc_mag_ae_prov_1min_1
in the same way

4. Click OK

GUI Basic Operation
ResultGUI Basic Operation
Lesson:Change X range (time scale)of the plot (1)Reduces X range
by major tickmarker
Expands X range
by major tickmarker
Shift left X range
by major tickmarker
Shift right X range
by major tickmarker

Use these icons

GUI Basic Operation
Lesson:Change X range (time scale) of theplot (2)1. Select
Plot – X Axis Options

GUI Basic Operation
1. Select Panel (If panel is locked,
use “Apply to All Panels”.)
2. Select Fixed Range

3. Change values
Min 2012-03-06/00:00:00.000
Max 2012-03-11/00:00:00.000
4. Click "Apply to All Panels”

GUI Basic OperationLesson:
Customize Ticks.

1. Select Major Ticks
By Number or Interval.
2. Input the number
into # (Number orInterval) of Minor Ticks

GUI Basic Operation
Lesson:Change Annotations

1. Select your favorite
format in the pull-downmenu of Annotation
Format.
2. If you want tochange the character
font, size, and color,select your favorite
format in the pulldown menu here.

3. Click "Apply to All Panels”

57

GUI Basic Operation
Lesson:Customize Labels (of X axis)

1. Select Panel 4(bottom panel)

3. Type “UniversalTime ” on the Edit/AddLabel

2. Check the“Show Label” box

4. Click "OK“.

GUI Basic Operation
ResultGUI Basic Operation
Other options.
• Page Options…
Customize the text and layout of thepage.
• Panel Options…
Customize the title and color ofeach panel.
• Line Options…
Customize the line and symbol ofeach plot panel.
• Legend Options…
Customize the legend whichappears when you put the mouse
cursor on the plot.
• Variable Options…
Display the values of the selectedparameters under the time label.

GUI Basic Operation
Lesson:Reset X range (time scale)
1. Select X Axis Options

GUI Basic Operation
1. Select (L) Panel 1(1, 1) 2. Select Auto Range

3. Click “OK”

GUI Basic Operation
Lesson:Processing- subtract average
1. SelectAnalysis – Data Processing

GUI Basic Operation
4. Click Subtract Average

3. Active Data are added
2. Click right arrow

1. Select data you want to process
magdas_mag_asb_1sec_hdz
magdas_mag_her_1sec_hdz

GUI Basic Operation

New variables are created.
magdas_mag_asb_1sec_hdz-d
magdas_mag_her_1sec_hdz-d

1. Click DoneGUI Basic Operation
Open “Plot/Layout Options”

1. Remove Panel 3 and 4
3. Click line
4. Data are added

2. Selectmagdas_mag_her_1sec_hdz-d

5. Then, add the other variable,
magdas_mag_asb_1sec_hdz-d
to panel in the same way.

6. Click OK

GUI Basic Operation

Subtracted average!

GUI Basic Operation
Try:
Plot Power Spectrum of
magdas_mag_her_1sec_hdz_x

Hint1: Use “Data Processing”
for calculating (if you will get
an option dialog, use defaultvalue)
Hint2: Use “Spec” for plotting

1. SelectAnalysis – Data Processing

GUI Basic Operation

2. Active Data is removed

1. Click Clear ActiveGUI Basic Operation

3. Active Data are added
2. Click right arrow

1. Select data
magdas_mag_her_1sec_hdz

5. Click OK

4. Click Power Spectrum

GUI Basic Operation

1. New variables are created!

2. Click DoneGUI Basic Operation
Open “Plot/Layout Options”

2. Click Spec
3. Data are added
1. Selectmagdas_mag_her_1sec_hdz_x_dpwrspc

4. Click OK

GUI Basic Operation
ResultThis color bar can becustomized inGraph - Z Axis
Options.

GUI Basic Operation
Lesson:Use Calculate (Equation editor)
1. SelectAnalysis – Calculate…

GUI Basic Operation
Equation Editor for SPEDAS
Variable=Your loaded data

Built-in function

Built-in Operator

Built-in Constant

GUI Basic Operation
Lesson:Make an equation using the loaded variables.

Type variable/function/Operator/Constant, and make equation
A = B + C - D…
Note:
Enclose the tplot variable in double quotation marks

GUI Basic Operation
Lesson:Make an equation using the loaded variables.
1. Select magdas_mag_her_1sec_hdz-d_x
3. Variable is added
2. Click arrow

Then, try to add the offset (+200) tomagdas_mag_her_1sec_hdz-d_x
and plot on new panel.

GUI Basic Operation
Note: one line, never return

Answer"magdas_mag_her_1sec_hdz-d_x_ofst" = "magdas_mag_her_1sec_hdz-d_x"+ 200

1. Click Run
2. A new variable is created
3. Click done

GUI Basic Operation
Open “Plot/Layout Options”

1. Select Panel 3 and Remove it

3. Click Line

4. Data are added

2. Selectmagdas_mag_her_1sec_hdz-d_y
magdas_mag_her_1sec_hdz-d_x_ofst

GUI Basic Operation
Open “Plot/Layout Options”

1. Change the value of Row to 3

2. Panel 6 is changed to (3, 1)

3. Click OKGUI Basic Operation
ResultOffset (+200nT)
was added to the Hcomp.

GUI Basic Operation
Try:
Expand the plot using the mouse.
By left-click anddrag the mouse

A new page opens

How to Use SPEDAS
part3
Additional data loading

GUI Basic Operation
Lesson:Load Additional ASCII Data

Sample1: magnetometer data

HEADER(13 lines)

Data(86400lines)

See Data/testfile_format0.txt

Data(6 columns)

GUI Basic Operation

1. Select
Data – Load Single File – Load ASCII…

GUI Basic Operation
1. Click”Browse”, and select‘testfile_format0.txt’.
2. Format Type: Select 0
3. Time Format: Check Specify, andput ‘YYYY-MM-DD hh:mm:ss.fff’
4. Column No. of loaded data:put ‘1,2,3,4’Note:Column number starts from 0.

5. Options for Header:
Check the box, and put ‘13’ toNumber of lines of skip.

6. Click OK

GUI Basic Operation

1. Click “OK”

GUI Basic Operation
Open “Plot/Layout Options”

2. Click Line

1. Click +, and selecttvar1_0

3. Data are added

4. Repeat the same process
to tvar1_1, tvar_1_2, andtvar1_3
5. Click “OK”

GUI Basic Operation
Magnetometer data
Click this icon before next lessonwritten inASCII(IAGA-2002)
format are plotted.

GUI Basic Operation
Lesson:Load External ASCII Data

Sample2: EISCAT radar data

HEADER(starts from %)Data(11520lines)

See Data/testfile_format1.txt

Data(19 columns)

GUI Basic Operation
Open File – Load Your Data – Load ASCII

1. Click”Browse”, and select‘testfile_format1.txt’.
2. Format Type: Select 1
3. Time Format: Check Specify, andput ‘YYYY-MM-DD hh:mm:ss.f’
4. Column No. of loaded data:put ‘5,6,7,8’
5. Loaded data name:put ‘Ne, Vi, Ti, Te’
6. Column No. of v_vector:
put ‘1’

5. Click OK

7. Options for Header:
Check the box, and put ‘%’ toComment symbolGUI Basic Operation

1. Click “OK”

GUI Basic Operation
Open “Plot/Layout Options”

2. Click Spec

3. Data are added
1. Select Ne
4. Repeat the sameprocess to Vi, Ti, and Te

5. Click “OK”

GUI Basic Operation

Click this icon before next lesson

EISCAT radar data written
in ASCII format are plottedin spectrogram.

GUI Basic Operation
Try:
Plot your own ASCII/CDF
data using Data – LoadSingle File – Load ASCII…

If you do not prepare yourown data, let’s try to plot
Data/practice.txt
Hint: Use format 1 for dataloading, and use SPEC fordata plotting

GUI Basic OperationAnswer
1. Click”Browse”, and select‘practice.txt’.

Open “Load ASCII”

2. Format Type: Select 1
3. Time Format: Check Specify, andput ‘YYYY-MM-DD/hh:mm:ss.fff’
4. Column No. of loaded data:put ‘1,2,3’
5. Loaded data name:put ‘u, v, w’
6. Delimiter: put ‘,’
7. Column No. of v_vector:
put ‘0’9. Click OK

8. Options for Header:
Check the box, and put ‘#’ toComment symbolGUI Basic Operation
Open “Plot/Layout Options”

1. Select u

3. Data are added
2. Click Spec

4. Repeat the sameprocess to v and w

Practice data(wind velocity
observed by MU radar)
written in ASCII format areplotted in spectrogram.

For advance…
UDAS website：http://www.iugonet.org/product/analysis.jsp

Acknowledgment
SPEDAS is a grass-roots data analysis software for the Space Physics community,
which was developed by scientists and programmers of the UC Berkeley's Space
Sciences Laboratory, UCLA's IGPP and other contributors

http://www.iugonet.org/Feedbacks
If you have any feedbacks, questions, requests
about this hands-on and software, please send
email to the following:
Subject: IUGONET hands-on @ PRIC
To: ytanaka@nipr.ac.jp

We would appreciate your comments!