Package: geospacelab
GeospaceLAB (geospacelab)GeospaceLAB provides a framework of data access, analysis, and visualization for the researchers in space physics and space weather. The documentation can be found
onreadthedocs.io.FeaturesClass-based data manager, includingDataHub: the core module (top-level class) to manage data from multiple sources,Dataset: the middle-level class to download, load, and process data from a data source,Variable: the base-level class to store the data array of a variable with various attributes, including its
error, name, label, unit, group, and dependencies.ExtendableProvide a standard procedure from downloading, loading, and post-processing the data.Easy to extend for a data source which has not been supported in the package.Flexible to add functions for post-processing.VisualizationTime series plots withautomatically adjustable time ticks and tick labels.dynamical panels (flexible to add or remove panels).automatically detect the time gaps.useful marking tools (vertical line crossing panels, shadings, top bars, etc, see Example 2 inUsage)Map projectionPolar views withcoastlines in either GEO or AACGM (APEX) coordinate system.mapping in either fixed lon/mlon mode or in fixed LST/MLT mode.Support 1-D or 2-D plots withsatellite tracks (time ticks and labels)nadir colored 1-D plotsgridded surface plotsSpace coordinate system transformationUnified interface for cs transformations.Toolboxes for data analysisBasic toolboxes for numpy array, datetime, logging, python dict, list, and class.Built-in data sources:Data SourceVariablesFile FormatDownloadableExpressStatusCDAWeb/OMNISolar wind and IMFcdfTrueOMNIDashboardstableMadrigal/EISCATIonospheric Ne, Te, Ti, ...EISCAT-hdf5,Madrigal-hdf5TrueEISCATDashboardstableMadrigal/GNSS/TECMAPIonospheric GPS TEC maphdf5True-betaMadrigal/DMSP/s1DMSP SSM, SSIES, etchdf5TrueDMSPTSDashboardstableMadrigal/DMSP/s4DMSP SSIEShdf5TrueDMSPTSDashboardstableMadrigal/DMSP/eDMSP SSJhdf5TrueDMSPTSDashboardstableMadrigal/Millstone Hill ISR+Millstone Hill ISRhdf5TrueMillstoneHillISRDashboardstableMadrigal/Poker Flat ISRPoker Flat ISRhdf5True_-stableJHUAPL/DMSP/SSUSIDMSP SSUSInetcdfTrueDMSPSSUSIDashboardstableJHUAPL/AMPERE/fittedAMPERE FACnetcdfFalseAMPEREDashboardstableSuperDARN/POTMAPSuperDARN potential mapasciiFalse-stableWDC/DstDst indexIAGA2002-ASCIITrue-stableWDC/ASYSYMASY/SYM indicesIAGA2002-ASCIITrueOMNIDashboardstableWDC/AEAE indicesIAGA2002-ASCIITrueOMNIDashboardstableGFZ/KpKp/Ap indicesASCIITrue-stableGFZ/HpoHp30 or Hp60 indicesASCIITrue-stableGFZ/SNF107SN, F107ASCIITrue-stableESA/SWARM/EFI_LP_HMSWARM Ne, Te, etc.netcdfTrue-stableESA/SWARM/EFI_TCT02SWARM cross track vinetcdfTrue-stableESA/SWARM/AOB_FAC_2FSWARM FAC, auroral oval boundarynetcdfTrue-betaTUDelft/SWARM/DNS_PODSwarm $\rho_n$ (GPS derived)ASCIITrue-stableTUDelft/SWARM/DNS_ACCSwarm $\rho_n$ (GPS+Accelerometer)ASCIITrue-stableTUDelft/GOCE/WIND_ACCGOCE neutral windASCIITrue-stableTUDelft/GRACE/WIND_ACCGRACE neutral windASCIITrue-stableTUDelft/GRACE/DNS_ACCGrace $\rho_n$ASCIITrue-stableTUDelft/CHAMP/DNS_ACCCHAMP $\rho_n$ASCIITrue-stableUTA/GITM/2DALLGITM 2D outputbinary,IDL-savFalse-betaUTA/GITM/3DALLGITM 3D outputbinary,IDL-savFalse-betaInstallation1. The python distribution "Anaconda" is recommended:The package was tested with the anaconda distribution and withPYTHON>=3.7underUbuntu 20.04andMacOS Big Sur.With Anaconda, it may be easier to install some required dependencies listed below, e.g., cartopy, using thecondacommand.
It's also recommended installing the package and dependencies in a virtual environment with anaconda.Afterinstalling the anaconda distribution, a virtual environment can be created by the code below in the terminal:condacreate--name[YOUR_ENV_NAME]-cconda-forgepythoncythoncartopyThe package "spyder" is a widely-used python IDE. Other IDEs, like "VS Code" or "Pycharm" also work.Note:The recommended IDE is Spyder. Sometimes, aRuntimeErrorcan be raised
when theaacgmv2package is called inPyCharmorVS Code.
If you meet this issue, try to compile the codes inSpyderseveral times.After creating the virtual environement, you need to activate the virtual environment:condaactivate[YOUR_ENV_NAME]and then to install the package as shown below or to start the IDESpyder.More detailed information to set the anaconda environment can be foundhere,2. InstallationQuick install from the pre-built release (recommended):pipinstallgeospacelabInstall fromGithub(not recommended):pipinstallgit+https://github.com/JouleCai/geospacelab@master2. DependenciesThe package dependencies need to be installed before or after the installation of the package.
Several dependencies will be installed automatically with the package installation,
includingtoml,requests,bueatifulsoup4,numpy,scipy,matplotlib,h5py,netcdf4,cdflib,madrigalweb,sscws, andaacgmv2.Other dependencies will be needed if you see aImportErrororModuleNotFoundErrordisplayed in the python console. Some frequently used modules and their installation methods are listed below:cartopy: Map projection for geospatial data.conda install -c conda-forge cartopyapexpy*: Apex and Quasi-Dipole geomagnetic
coordinate system.pip install apexpygeopack: The geopack and Tsyganenko models in Python.pip install geopack(*): Thegccorgfortrancompilers are required before installing the package.gcc:conda install -c conda-forge gccgfortran:conda install -c conda-forge gfortranPlease install the packages above, if needed.Note: The package is currently pre-released. The installation methods may be changed in the future.4. First-time startup and basic configurationSome basic configurations will be made with the first-time import of the package. Following the messages prompted in the python console, the first configuration is to set the root directory for storing the data.When the modules to access the online Madrigal database is imported,  it will ask for the inputs of user's full name, email, and affiliation.The user's configuration can be found from thetomlfile below:[your_home_directory]/.geospacelab/config.tomlThe user can set or change the preferences in the configuration file. For example, to change the root directory for storing the data, modify or add the lines in "config.toml":[datahub]data_root_dir="YOUR_ROOT_DIR"To set the Madrigal cookies, change the lines:[datahub.madrigal]user_fullname="YOUR_NAME"user_email="YOU_EMAIL"user_affiliation="YOUR_AFFILIATION"5. UpgradeIf the package is installed from the pre-built release. Update the package via:pipinstallgeospacelab--upgrade6. UninstallationUninstall the package via:pipuninstallgeospacelabIf you don't need the user's configuration, delete the file at[your_home_directory]/.geospacelab/config.tomlUsageExample 1: Dock a sourced dataset and get variables:The core of the data manager is the class Datahub. A Datahub instance will be used for docking a buit-in sourced dataset, or adding a temporary or user-defined dataset.The "dataset" is a Dataset instance, which is used for loading and downloading
the data.Below is an example to load the EISCAT data from the online service.  The module will download EISCAT data automatically fromthe EISCAT schedule pagewith the presetttings of loading mode "AUTO" and file type "eiscat-hdf5".Example 1:importdatetimefromgeospacelab.datahubimportDataHub# settingsdt_fr=datetime.datetime.strptime('20210309'+'0000','%Y%m%d%H%M')# datetime fromdt_to=datetime.datetime.strptime('20210309'+'2359','%Y%m%d%H%M')# datetime todatabase_name='madrigal'# built-in sourced database namefacility_name='eiscat'# facility namesite='UHF'# facility attributes required, check from the eiscat schedule pageantenna='UHF'modulation='ant'# create a datahub instancedh=DataHub(dt_fr,dt_to)# dock the first dataset (dataset index starts from 0)ds_isr=dh.dock(datasource_contents=[database_name,'isr',facility_name],site=site,antenna=antenna,modulation=modulation,data_file_type='eiscat-hdf5')# load datads_isr.load_data()# assign a variable from its own dataset to the datahubn_e=dh.assign_variable('n_e')T_i=dh.assign_variable('T_i')# get the variables which have been assigned in the datahubn_e=dh.get_variable('n_e')T_i=dh.get_variable('T_i')# if the variable is not assigned in the datahub, but exists in the its own dataset:comp_O_p=dh.get_variable('comp_O_p',dataset=ds_isr)# O+ ratio# above line is equivalent tocomp_O_p=dh.datasets[0]['comp_O_p']# The variables, e.g., n_e and T_i, are the class Variable's instances,# which stores the variable values, errors, and many other attributes, e.g., name, label, unit, depends, ....# To get the value of the variable, use variable_isntance.value, e.g.,print(n_e.value)# return the variable's value, type: numpy.ndarray, axis 0 is always along the time, check n_e.depends.items{}print(n_e.error)Example 2: EISCAT quicklook plotThe EISCAT quicklook plot shows the GUISDAP analysed results in the same format as the online EISCAT quicklook plot.
The figure layout and quality are improved. In addition, several marking tools like vertical lines, shadings, top bars can be
added in the plot. See the example script and figure below:In "example2.py"importdatetimeimportgeospacelab.express.eiscat_dashboardaseiscatdt_fr=datetime.datetime.strptime('20201209'+'1800','%Y%m%d%H%M')dt_to=datetime.datetime.strptime('20201210'+'0600','%Y%m%d%H%M')site='UHF'antenna='UHF'modulation='60'load_mode='AUTO'dashboard=eiscat.EISCATDashboard(dt_fr,dt_to,site=site,antenna=antenna,modulation=modulation,load_mode='AUTO')dashboard.quicklook()# dashboard.save_figure() # comment this if you need to run the following codes# dashboard.show()   # comment this if you need to run the following codes."""As the dashboard class (EISCATDashboard) is a inheritance of the classes Datahub and TSDashboard.The variables can be retrieved in the same ways as shown in Example 1."""n_e=dashboard.assign_variable('n_e')print(n_e.value)print(n_e.error)"""Several marking tools (vertical lines, shadings, and top bars) can be added as the overlayson the top of the quicklook plot."""# add vertical linedt_fr_2=datetime.datetime.strptime('20201209'+'2030',"%Y%m%d%H%M")dt_to_2=datetime.datetime.strptime('20201210'+'0130',"%Y%m%d%H%M")dashboard.add_vertical_line(dt_fr_2,bottom_extend=0,top_extend=0.02,label='Line 1',label_position='top')# add shadingdashboard.add_shading(dt_fr_2,dt_to_2,bottom_extend=0,top_extend=0.02,label='Shading 1',label_position='top')# add top bardt_fr_3=datetime.datetime.strptime('20201210'+'0130',"%Y%m%d%H%M")dt_to_3=datetime.datetime.strptime('20201210'+'0430',"%Y%m%d%H%M")dashboard.add_top_bar(dt_fr_3,dt_to_3,bottom=0.,top=0.02,label='Top bar 1')# save figuredashboard.save_figure()# show on screendashboard.show()Output:Example 3: OMNI data and geomagnetic indices (WDC + GFZ):In "example3.py"importdatetimeimportgeospacelab.express.omni_dashboardasomnidt_fr=datetime.datetime.strptime('20160314'+'0600','%Y%m%d%H%M')dt_to=datetime.datetime.strptime('20160320'+'0600','%Y%m%d%H%M')omni_type='OMNI2'omni_res='1min'load_mode='AUTO'dashboard=omni.OMNIDashboard(dt_fr,dt_to,omni_type=omni_type,omni_res=omni_res,load_mode=load_mode)dashboard.quicklook()# data can be retrieved in the same way as in Example 1:dashboard.list_assigned_variables()B_x_gsm=dashboard.get_variable('B_x_GSM',dataset_index=0)# save figuredashboard.save_figure()# show on screendashboard.show()Output:Example 4: Mapping geospatial data in the polar map.importdatetimeimportmatplotlib.pyplotaspltimportgeospacelab.visualization.mpl.geomap.geodashboardsasgeomapdt_fr=datetime.datetime(2015,9,8,8)dt_to=datetime.datetime(2015,9,8,23,59)time1=datetime.datetime(2015,9,8,20,21)pole='N'sat_id='f16'band='LBHS'# Create a geodashboard objectdashboard=geomap.GeoDashboard(dt_fr=dt_fr,dt_to=dt_to,figure_config={'figsize':(5,5)})# If the orbit_id is specified, only one file will be downloaded. This option saves the downloading time.# dashboard.dock(datasource_contents=['jhuapl', 'dmsp', 'ssusi', 'edraur'], pole='N', sat_id='f17', orbit_id='46863')# If not specified, the data during the whole day will be downloaded.dashboard.dock(datasource_contents=['jhuapl','dmsp','ssusi','edraur'],pole=pole,sat_id=sat_id,orbit_id=None)ds_s1=dashboard.dock(datasource_contents=['madrigal','satellites','dmsp','s1'],dt_fr=time1-datetime.timedelta(minutes=45),dt_to=time1+datetime.timedelta(minutes=45),sat_id=sat_id)dashboard.set_layout(1,1)# Get the variables: LBHS emission intensiy, corresponding times and locationslbhs=dashboard.assign_variable('GRID_AUR_'+band,dataset_index=0)dts=dashboard.assign_variable('DATETIME',dataset_index=0).value.flatten()mlat=dashboard.assign_variable('GRID_MLAT',dataset_index=0).valuemlon=dashboard.assign_variable('GRID_MLON',dataset_index=0).valuemlt=dashboard.assign_variable(('GRID_MLT'),dataset_index=0).value# Search the index for the time to plot, used as an input to the following polar mapind_t=dashboard.datasets[0].get_time_ind(ut=time1)lbhs_=lbhs.value[ind_t]mlat_=mlat[ind_t]mlon_=mlon[ind_t]mlt_=mlt[ind_t]# Add a polar map panel to the dashboard. Currently the style is the fixed MLT at mlt_c=0. See the keywords below:panel1=dashboard.add_polar_map(row_ind=0,col_ind=0,style='mlt-fixed',cs='AACGM',mlt_c=0.,pole=pole,ut=time1,boundary_lat=65.,mirror_south=True)# Some settings for plotting.pcolormesh_config=lbhs.visual.plot_config.pcolormesh# Overlay the SSUSI image in the map.ipm=panel1.overlay_pcolormesh(data=lbhs_,coords={'lat':mlat_,'lon':mlon_,'mlt':mlt_},cs='AACGM',regridding=True,**pcolormesh_config)# Add a color barpanel1.add_colorbar(ipm,c_label=band+" (R)",c_scale=pcolormesh_config['c_scale'],left=1.1,bottom=0.1,width=0.05,height=0.7)# Overlay the gridlinespanel1.overlay_gridlines(lat_res=5,lon_label_separator=5)# Overlay the coastlines in the AACGM coordinatepanel1.overlay_coastlines()# Overlay cross-track velocity along satellite trajectorysc_dt=ds_s1['SC_DATETIME'].value.flatten()sc_lat=ds_s1['SC_GEO_LAT'].value.flatten()sc_lon=ds_s1['SC_GEO_LON'].value.flatten()sc_alt=ds_s1['SC_GEO_ALT'].value.flatten()sc_coords={'lat':sc_lat,'lon':sc_lon,'height':sc_alt}v_H=ds_s1['v_i_H'].value.flatten()panel1.overlay_cross_track_vector(vector=v_H,unit_vector=1000,alpha=0.5,color='r',sc_coords=sc_coords,sc_ut=sc_dt,cs='GEOC')# Overlay the satellite trajectory with tickspanel1.overlay_sc_trajectory(sc_ut=sc_dt,sc_coords=sc_coords,cs='GEOC')# Add the title and save the figurepolestr='North'ifpole=='N'else'South'panel1.add_title(title='DMSP/SSUSI, '+band+', '+sat_id.upper()+', '+polestr+', '+time1.strftime('%Y-%m-%d%H%M UT'))plt.savefig('DMSP_SSUSI_'+time1.strftime('%Y%m%d-%H%M')+'_'+band+'_'+sat_id.upper()+'_'+pole,dpi=300)# show the figureplt.show()Output:This is an example showing the HiLDA aurora in the dayside polar cap region
(see alsoDMSP observations of the HiLDA aurora (Cai et al., JGR, 2021)).Other examples for the time-series plots and map projections can be foundhereAcknowledgements and CitationAcknowledgementsWe acknowledge all the dependencies listed above for their contributions to implement a lot of functionality in GeospaceLAB.CitationIf GeospaceLAB is used for your scientific work, please mention it in the publication and cite the package:Cai L, Aikio A, Kullen A, Deng Y, Zhang Y, Zhang S-R, Virtanen I and Vanhamäki H (2022), GeospaceLAB: Python package
for managing and visualizing data in space physics. Front. Astron. Space Sci. 9:1023163. doi:10.3389/fspas.2022.1023163In addition, please add the following text in the "Methods" or "Acknowledgements" section:This research has made use of GeospaceLAB v?.?.?, an open-source Python package to manage and visualize data in space physics.Please include the project logo (see the top) to acknowledge GeospaceLAB in posters or talks.Co-authorshipGeospaceLAB aims to help users to manage and visualize multiple kinds of data in space physics in a convenient way. We welcome collaboration to support your research work. If the functionality of GeospaceLAB plays a critical role in a research paper, the co-authorship is expected to be offered to one or more developers.NotesThe current version is a pre-released version. Many features will be added soon.