SPEDAS training session– Beginner course –T. Hori, M. Teramoto, T.-F. Chang, Y. Tsugawa, M. Shoji,
S. Kurita, Y. Miyoshi, N. Umemura, T. Segawa(ERG Science Center, ISEE, Nagoya Univ.)

1

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Purpose of this course} To get familiar with the SPEDAS scheme.
} To get you ready to use SPEDAS to analyze ERG

data.

2

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

What will you be doing in this training?
} Learn about the data model employed by SPEDAS.
} Load/manipulate/visualize ERG satellite and otherground data.

3

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Keep in mind
} This is a "hands-on" training for SPEDAS, not the timefor e-mail check.

} Communicate with lecturers, tutors, neighboringskilled users.

} The training session today does not cover everything

in the handouts due to time limitation. It is
recommended to practice the uncovered parts later by
yourself.

} We will proceed slowly with entry-level SPEDAS users,

particularly in the beginner course. You can go fasterby yourself.

4

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

What's SPEDAS?

5SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Space Physics Environment Data Analysis
Software (SPEDAS)
SPEDAS frameworkDatarepository

CDF

CDF

CDF
Van Allen
Probes

CDF

ERG satellite
data
Each project
develops andreleases APIs toautomaticallydownload and readeach project data

CDF

ERG-related groundobservation
network data
CDF
HDF?

CDF

ERG simulation/ modeling data
ERG-SC develops dataarchives and APIs to readthe data from SPEDAS

6SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

What can you do with SPEDAS?
} time-series plots} filtering of data
} frequency analysis} mapping to the ground maps
} …

7

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Basics of SPEDAS:
tplot and tplot variable

8

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

A few basics of IDL before entering SPEDAS…
Insert a comma ( , ) between a command, its arguments, and keywords.

}

IDL> tplot , 1 , title='New plot'

A string is expressed as a text sandwiched by delimeters ( ' ) or ( " ).

}IDL> print, 'This is a text.'

An array is expressed as comma-separated elements that are bracketed.

}

IDL> arr1 = [ 2, 3, 4, 5 ]IDL> string_arr1 = [ 'text1', 'text2', 'text3' ]Typical errors beginners often encounter:}% Attempt to call undefined procedure: '????'.
à command/routine name (????) is misspelled.
% Syntax error.
à , ' ( ) [ ] is missing or mismatched in most cases.

Use Up arrow key to reuse previously typed commands. You can edit
them with Left/Right arrow, Backspace keys and execute!

}

9

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

How SPEDAS works?
One of the simplest procedures would be:
Run IDL
Initialize the SPEDAS environment on IDL
Set a date/time range for which data are loaded.
Load data

1.
2.
3.
4.

(Manipulate the loaded data)

5.

Plot the data

6.

10

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

How SPEDAS works?
In SPEDAS command lines,
prompt> idl
IDL> erg_init
ERG> timespan, '2017-06-16', 3
ERG> thm_load_gmag, site='atha pokr'(manipulate tplot variables)

ERG> tplot, [ 'thg_mag_atha', 'thg_mag_pokr' ]11

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Set a date/time range
ERG> timespan, timestr, N, option
timestr : a string expressing a particular date/timein UTC in the format of 'yyyy-mm-dd/hh:mm:ss'
N : number of time length (Default: 1)
option : unit (/day, /hour, /min, /sec, Default: /day)

For 1 day from 2017-06-16/00:00:00 UTC
ERG> timespan, '2017-06-16'
For 10 min from 2017-06-17/01:31:41 UTC
ERG> timespan, '2017-06-17/01:31:41', 10, /min
12

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Tplot variable as the primary data model} 'thg_mag_atha' in prev. page is called tplot variable.
} "Tplot variables" bind an indexed name-string to adata structure on IDL containing time-series datawith metadata.
SPEDAS session#1 'tplot var. name 1'#2 'tplot var. name 2'

13

Memory space of IDL
timeseries
data 1

Time
array

Data
array

metadata

timeseries
data 2

Time
array

Data
array

metadata

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Listing tplot data & viewing the content
ERG> tplot_names
ERG> print_tinfo, 'thg_mag_pokr'

All tplot variables
are listed withunique index
numbers

14

The actual data structure bound to tplot variable'thg_mag_pokr' is shown.

X: time array containing time labels in decimal UNIX time
Y: data array, in this case, a 2-D array of time x 3-components
V: array containing indices of the 3-components, [ 1 , 2 , 3 ]SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Listing tplot data & viewing the content
ERG> tplot_names, 'thg_mag_atha', /verbose
ERG> tplot_names, 1, /v

Metadata (informationon the data) are dumped.
RULES_OF_USE carries"rules of the road" inusing the data.
15

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Plotting a tplot data by tplot
Tplot with tplot variable names (string)
ERG> tplot, 'thg_mag_atha'
ERG> tplot, [ 'thg_mag_atha' , 'thg_mag_pokr' ]Tplot with the index number of a tplot variable
ERG> tplot, 1
With an array of variable indices to combine multiple variable in a single plot window
ERG> tplot, [2,1]Some wildcards can be used
ERG> tplot, 'thg_mag_*'

Tplot accepts variables asarguments in various formats.

16

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Decorate the plot panel for each tplot variableoptions, varname, option1='…', option2='…', …
varname: tplot variable name (wildcards accepted)
option?: name of tplot variable attribute

ERG> options, 'thg_mag_atha', ytitle='ATHA geomag' , ysubtitle='[nT]'
ERG> options, 'thg_mag_????', labels=[ 'H-comp', 'D-comp', 'Z-comp' ]ERG> tplot, 'thg_mag_atha'

ytitle,ysubtitlelabels17

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Separate a tplot variable with vector data
ERG> split_vec, 'thg_mag_atha'
STORE_DATA(264): Creating tplot variable: 3 thg_mag_atha_x
STORE_DATA(264): Creating tplot variable: 4 thg_mag_atha_y
STORE_DATA(264): Creating tplot variable: 5 thg_mag_atha_z
ERG> tplot, 'thg_mag_atha_?'split_vec takes a tplot variablewith vector or array data tocreate new tplot variables
containing each component of
the vector/array data.

18

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Change the time range of a plot
Select a time period by mouse-clicks on the plot window
ERG> tlimit
Specify a time period explicitly

ERG> tlimit, '2017-06-16/00:00' , '2017-06-17/00:00'
Back to the last plot period
ERG> tlimit, /last

Restore the original plot periodthat was set by timespan
ERG> tlimit, /full

19

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Change the vertical scale of a plot
ylim, varname, ymin, ymax, logflag

varname : variable name(s)ymin/ymax : lower/upper limit along vertical axis
set both to 0 (zero) for plotting with auto-scale
logflag : set 0 (zero) for plotting on a linear scale, or 1 for a log scale

ERG> ylim, 'thg_mag_atha_x', 13950,14050,
ERG> tplot

0

Zoomed in a more limited
range in the vertical scale.

Tips:
Putting 0 for both ymin and ymax
sets the y range to auto-scale.
ERG> ylim, 'thg_mag_atha_x', 0, 0

20

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Dump to png, postscript, and Ascii files
To a png file or postscript file
ERG> cwd;Display the current directory
CWD(25): Directory changed to: /yyyy/xxxx
ERG> tplot, 'thg_mag_atha_?'
ERG> makepng, 'atha_plot';à atha_plot.png/yyyy/xxxx/atha_plot.png

ERG> popen, 'atha_plot'
ERG> tplot;Redo the last plot
ERG> pclose; à atha_plot.ps

Dump the data content of a tplot variable to a Ascii file
ERG> tplot_ascii, 'thg_mag_atha';--> thg_mag_atha.txt, thg_mag_atha_v.txt

21

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

SPEDAS manual viewed by web browsers}

Open/YOUR_SPEDAS_DIR/idl/_spd_doc.html

with your web browser to view
the automatically generated
documents for SPEDAS
routines.

22

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Basics of SPEDAS:
Various filtering routines for tplot data

23

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Subtract the mean level – tsub_average –tsub_average, 'varname'varname: tplot variable names or index numbers
ERG> del_data, '*'Remove all tplot variables and reload the data
ERG> timespan, '2017-06-16'
ERG> thm_load_gmag, site='atha'
ERG> tsub_average, 'thg_mag_atha'
ERG> tplot, ['thg_mag_atha', 'thg_mag_atha-d' ]24

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

boxcar-average data – avg_data –avg_data, 'varname', timebin
varname: tplot variable names or index numbers
timebin : a time window in sec with which the boxcar-averaging is applied to the data
ERG> avg_data, 'thg_mag_atha-d', 1000.
STORE_DATA(264): Creating tplot variable: 3 thg_mag_atha-d_avg

ERG> tplot, 'thg_mag_atha-d*'

The data boxcaraveraged with a timebin of 1000 second
As a result, the number of data points
is reduced to every 1000 s.

25

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Smoothing data – tsmooth_in_time –tsmooth_in_time, 'varname', timebin
varname : tplot variable name(s)timebin : time window in second for running average

ERG> tsmooth_in_time, 'thg_mag_atha-d', 1000.
ERG> tplot_names… … … …
2 tha_mag_atha-d… … … …
4 tha_mag_atha-d_smoothed
ERG> tplot, [ 2 , 4 ]The data is running-averaged with atime window of 1000 second. We canuse this as a rough low-pass filter.
Note that the number of data points
is conserved.
26

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

High-pass filter in time – thigh_pass_filter –thigh_pass_filter, 'varname', timebin
varname : tplot variable name(s)timebin : time window in second for running average
ERG> thigh_pass_filter, 'thg_mag_atha-d', 1000.
ERG> tplot_names… … … …
2 tha_mag_atha-d… … … …
5 tha_mag_atha-d_hpfilt
ERG> tplot, [ 2 , 5 ]Time variations with periods shorter
than 1000 sec are shown.
Actually this command just subtracts
the low-pass-filtered values derived
with tsmooth_in_time from theoriginal data, not uses any digital
filtering process such as FFT.
27

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Basics of SPEDAS:
Frequency analysis of tplot data

28

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Dynamics spectra – tdpwrspc–
tdpwrspc, 'varname'varname : tplot variable name(s)ERG> tdpwrspc, 'thg_mag_atha-d'
ERG> tplot_names… … … …
5 tha_mag_atha-d_hpfilt… … … …
9 thg_mag_atha-d_x_dpwrspc
ERG> tplot, [ 5, 9 ]FFT with the hanning window
is applied to derive dynamic
frequency spectra of the data.

29

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Wavelet analysis – wav_data –wav_data, 'varname'varname : tplot variable name(s)wav_data accepts data with less than 32768
ERG> split_vec, 'thg_mag_atha-d'samples. The number of data points is
ERG> avg_data, 'thg_mag_atha-d_x', 5.
reduced as done with avg_data in this case.
ERG> wav_data, 'thg_mag_atha-d_x_avg'
STORE_DATA(264): Creating tplot variable: 13 thg_mag_atha-d_x_avg_wv_pow
ERG> tplot, 'thg_mag_atha-d_x_avg*'
ERG> zlim, 13, 100, 1000000, 1
ERG> tplot, 'thg_mag_atha-d_x_avg*'

Wavelet analysis is applied toderive dynamic spectra of the data.
zlim is similar to "ylim"
command, but set the lower/upper
limit of the color scale for aspectrum-type plot.

30

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Other information sources for SPEDAS}SPEDAS wiki}

http://spedas.org/wiki/index.php?title=Main_Page}}

Change log of the source repository for the bleeding edge of SPEDAS}}

User's guide, Plug-in developer's guide, tips and tricks, The list of available crib sheets,…

http://spedas.org/changelog/

Crib sheets for TPLOT in Your_SPEDAS_dir/idl/general/examples/
}
}
}
}
}
}

31

crib_tplot.pro -- basic tplot intro
crib_tplot_annotation.pro -- How to control annotations in tplot (labels, text,
etc...)
crib_tplot_export_print.pro -- How to export tplot data and tplot plots
crib_tplot_layout.pro -- How to control tplot plot layouts
crib_tplot_range.pro -- How to control the range and scaling of tplot plots
crib_tplot_ticks.pro -- How to control tplot plot ticks. (location, size, etc...)SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Sample cases of ERG data analysis

32

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Data analysis:
3-day plot of XEP data and geomag. indices
ERG> timespan, '2017-06-16', 3, /dayLoad HEP data

ERG> erg_load_hep, datatype='omniflux'
Load other index data

ERG> omni_hro_load, /res5min
ERG> tplot,[ 'OMNI_HRO_5min_SYM_H' , 'OMNI_HRO_5min_AL_INDEX' ,'erg_hep_l2_FEDO_*' ]33

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Onboard instrument data:

Coordinate transformation of MGF data
Coordinate transformation (SGA, SGI, DSI, J2000)
From J2000 to the geophysical coordinates -------> using “cotrans”

ERG> timespan, '2017-03-28' & erg_load_orb & erg_load_mgf
ERG> erg_cotrans, 'erg_mgf_l2_mag_8sec_dsi', 'mgf_j2000',in_coord='dsi', out_coord='j2000'
Coordinate transformation (SM<-->J2000)

ERG> spd_cotrans, 'mgf_j2000', 'mgf_gsm', in_coord='j2000', out_coord='gsm'
ERG> tplot, ['mgf_gsm', 'erg_orb_l2_pos_blocal']

34

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Data analysis:
Frequency analysis of MGF data
Please prepare a tplot variable "mgf_sm" containing the magnetic field
vectors in SM coordinates for March 28 (hint: See the prev. page!)
ERG> timespan, '2017-03-28', 1, /dayDynamic spectra by FFT with the Hanning window
ERG> tdpwrspc, 'mgf_sm'
ERG> wav_data, 'mgf_sm_z' Dynamic spectra by a wavelet analysis
ERG> tplot,'mgf_sm_z*'

Use tlimit to zoom in aperiod between perigee
points!

35

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Onboard instrument data:
Level-2 particle omni-flux data;; Set the time spantimespan, '2017-04-19';; Load data
erg_load_xep, datatype='omniflux'
erg_load_hep, datatype='omniflux'
erg_load_mepe, datatype='omniflux'
erg_load_lepe, datatype='omniflux'
erg_load_mepi_nml, datatype='omniflux'
erg_load_lepi_nml, datatype='omniflux'
tplot_names
tplot,[ 'erg_xep_l2_FEDO_SSD', $'erg_hep_l2_FEDO_H','erg_hep_l2_FEDO_L', $'erg_mepe_l2_omniflux_FEDO', $'erg_lepe_l2_omniflux_FEDO', $'erg_mepi_l2_omniflux_FPDO', $'erg_lepi_l2_omniflux_FPDO'36]

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Change the time range of a plot: tlimit
Select a time period by mouse-clicks on the plot window
ERG> tlimit
Specify a time period explicitly

ERG> tlimit, '2017-04-19/00:00' , '2017-04-19/08:00'
Back to the last plot period
ERG> tlimit, /last

Restore the original plot periodthat was set by timespan
ERG> tlimit, /full

37

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Onboard instrument data:
Add MGF data with /add keyword
Load MGF data and add it to the pre-existing particle plot

ERG> erg_load_mgf, uname=uname, pass=pass
ERG> tplot, 'erg_mgf_l2_mag_8sec_sm' , /add

38

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Orbit data:
Set time range and load ERG orbit data
Setup the time range (‘YYYY-MM-DD/hh:mm:ss’)

ERG> timespan,'2017-03-28/00:00:00’,3,/day
Load orbit data

ERG> erg_load_orb
ERG> tplot_names("erg_orb_l2" means ERG Level-2 orbit data)

39(*) Using the IGRF model

pos_gse/gsm/sm

s/c position [Re] in GSE, GSM, SM coordinates

pos_rmlatmlt

Radial distance [Re], magnetic latitude [deg], local time [hr]of s/c position

pos_eqs/c position mapped to the magnetic equator

pos_iono_north/south

Geographic latitude and longitude [deg] of s/c footprints at
100 km altitude in the northern/southern hemisphere

pos_blocal / blocal_mag

model B-field vector (blocal) and B-field strength(blocal_mag) [nT] at s/c position

pos_beq / beq_mag

model B-field vector (beq) and B-field strength (beq_mag)[nT] at s/c position mapped to the magnetic equator

pos_Lm

McIlwain's L-parameter of s/c position for pitch angles of90, 60, and 30 deg

vel_gse/gsm/sm

s/c orbital velocity [km/s] in GSE, GSM, and SM

man_prep/man_on/eclipse_flag

flag for maneuver preparation (man_prep), maneuveron/off (man _on), and solar eclipse (eclipse)

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Orbit data:
Definitive orbit as a time series plot: tplot,

tplotxy

Plot orbit time series data

ERG> tplot, 'erg_orb_l2_pos_gsm'
Plot orbit data in the X-Y plane.

ERG> tplotxy, 'erg_orb_l2_pos_gsm'

40

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Separate a tplot variable with vector data:

split_vec

ERG> split_vec, 'erg_orb_l2_pos_rmlatmlt'
STORE_DATA(264): Creating tplot variable: ...
STORE_DATA(264): Creating tplot variable: ...
STORE_DATA(264): Creating tplot variable: ...
ERG> tplot, 'erg_orb_l2_pos_rmlatmlt_?'

split_vec takes a tplot variablewith vector or array data tocreate new tplot variables
containing each component of
the vector/array data.

41

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Orbit data:
Insert orbit values below a time-series plot
ERG> set_erg_var_label
ERG> tplot;just type "tplot" to replot the previous panels

Using subroutine to add labels(Lm,MLT,MLAT) with time

ERG> split_vec, 'erg_orb_l2_pos_rmlatmlt'
ERG> options, 'erg_orb_l2_pos_rmlatmlt_x', ytitle='R'
ERG> options, 'erg_orb_l2_pos_rmlatmlt_y', ytitle='MLAT'
ERG> options, 'erg_orb_l2_pos_rmlatmlt_z', ytitle='MLT'
ERG> tplot_options, var_label=['erg_orb_l2_pos_rmlatmlt_z','erg_orb_l2_pos_rmlatmlt_y', 'erg_orb_l2_pos_rmlatmlt_x']
ERG> tplot(Lm, MLAT, MLT)

42

An example to add
new labels(R,MLAT,MLT)(R, MLT, MLAT)SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Appendix 1:
Access the data structure in a tplot variable andcreate a new tplot variable (get_data, store_data)

43

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Access the data structure in a tplot variable– get_data –ERG> get_data, 'thg_mag_atha', data=data, dlimits=dlimits, lim=lim
data: the data structure is stored
dlimits: most of metadata are stored
lim: some plot properties are stored
ERG> help, data.x , data.y"get_data" extracts the data
structure of a tplot variable andsaves in a structure "data" of IDL
session in the above case, so thatusers can access them by referring
to as "data.x" or "data.y", forexample.
44

The information on the original CDF
data file (CDF) and metadata, andvarious plot properties are extracted
into a structure "dlimits".

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Create a new tplot variable– store_data –ERG> store_data, 'varname' , data = { x:timearr, y:datarr }varname: name of a newly created tplot variabletimearr: 1-D array containing time values in SPEDAS time of time-series datadatarr: 1-D or 2-D array containing the data values of time-series data. The size of 1st
dimension should be identical to that of timearr.

}}}

SPEDAS time is the UNIX time in double-precision floating-point values. UNIX timeis the elapsed second since 00:00 UTC on January 1, 1970.
Usually we use time_double() function to calculate a SPEDAS time value from atime string such as '2017-06-16/12:30:00'.
SPEDAS time values can easily be converted to time strings with time_string()
function.

Please refer to the SPEDAS wiki at
http://spedas.org/wiki/index.php?title=Time_handlingfor more details of the time handling in SPEDAS.

45

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Appendix 2:
Various options for SPEDAS use

46

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018

Options for SPEDAS use}

Using SPEDAS source code
}
}
}
}

}

Using SPEDAS save file
}
}
}
}

}

binary files working with the IDL virtual machine (IDL-VM) environment
IDL-VM can be downloaded for free from Exelis/Harris Geospatial.
No command-line tools: only GUI is available.
only plug-ins implemented to the save file are available for use. Somelatest plug-ins may not be included yet.

Using SPEDAS executable files}}

All SPEDAS IDL source code including both the command-line tools
and graphical user interface (GUI) tools.
To use it, you need to have IDL installed with a proper license.
The latest plug-ins can be used by installing by yourself.
Users can develop, edit, and run their own scripts.

A package bundling IDL-VM and the SPEDAS save file, basicallyequivalent to the above.

We use the source code version in this training session.
47

SPEDAS training session, ERG particle data WS @Tohoku Univ.

Sep. 20, 2018