TestingExecutorSiwenna

Instructions for installing and configuring a WMT executor on siwenna.
--Mpiper (talk) 12:33, 22 February 2018 (MST)
==Set install directory==
The install directory for this executor is /home/csdms/wmt/_testing.
 install_dir=/home/csdms/wmt/_testing
 mkdir -p $install_dir
==Install Python==
==Install the CSDMS software stack==
Using the csdms-stack conda channel (the Bakery)
install the CSDMS software stack,
including several pre-built components,
with the `csdms-stack` metapackage.
 conda install csdms-stack netcdf-fortran=4.4.4=5 -c csdms-stack -c defaults -c conda-forge
This metapackage currently includes
* pymt
* cca-tools
* csdms-child
* csdms-sedflux-3d
* csdms-hydrotrend
* csdms-permamodel-ku
* csdms-permamodel-frostnumber
* csdms-permamodel-kugeo
* csdms-permamodel-frostnumbergeo
* csdms-cruaktemp
* csdms-brake
* csdms-pydeltarcm
As of 2018-03-20, wind back the `netcdf-fortran` package so that HDF5 1.8.18 is used. There are problems with HDF5 1.10.1 and ESMF, used in PyMT. Hopefully, this will be a temporary change.
Optionally install the `babelizer`,
in case a component needs to be built from source.
 conda install -c csdms-stack babelizer
Optionally install IPython for testing.
 conda install ipython
Note that when running IPython remotely on siwenna,
it's helpful to set
 export MPLBACKEND=Agg
===Install executor software===
Install the `wmt-exe` package from source.
 mkdir -p $install_dir/opt && cd $install_dir/opt
 git clone https://github.com/csdms/wmt-exe
 cd wmt-exe
 python setup.py develop
Create a site configuration file that describes the executor and symlink it to the executor's etc/ diectory.
 python setup.py configure --wmt-prefix=$install_dir --launch-dir='/scratch/$USER/wmt' --exec-dir='/scratch/$USER/wmt'
 ln -s "$(realpath wmt.cfg)" $install_dir/conda/etc
Here I've configured the executor to do stuff in the /scratch directory instead of the default ~/.wmt.
==Install and test CSDMS components==
Each section below
describes how to install and test a particular CSDMS component.
Consider setting the PYMT_DEBUG environment variable.
 export PYMT_DEBUG=True
It's really helpful for debugging.
Note that the ILAMB component from the Bakery
won't work because it's built against `pymt=0.2`;
in particular,
my BMI for ILAMB is incompatible
with the `model_metadata` package in newer versions of PyMT.
Currently installed components:
# BRaKE
# Child (no wmt-metadata)
# CMIP
# CRUAKTemp
# FrostNumberGeoModel
# FrostNumberModel
# Hydrotrend
# KuGeoModel
# KuModel
# PyDeltaRCM
# Sedflux3D (no wmt-metadata)
===HydroTrend===
Hydrotrend is included in the `csdms-stack` metapackage.
Test it by starting a Python session
in $install_dir/test
and executing the setup and IRF methods.
 from pymt.components import Hydrotrend
 
 comp = Hydrotrend()
 comp.get_component_name()
 # args = comp.setup('.')
 # comp.initialize(*args)
 comp.setup('.')
 comp.initialize(None)
 comp.get_start_time()
 comp.get_end_time()
 comp.get_current_time()
 comp.update()
 comp.finalize()
===Permamodel and CRUAKTemp===
Permamodel and CRUAKTemp are included in the `csdms-stack` metapackage.
Test a component by starting a Python session
in $install_dir/test
and executing the setup and IRF methods.
 from pymt.components import FrostNumberModel
 
 comp = FrostNumberModel()
 comp.get_component_name()
 # args = comp.setup('.')
 # comp.initialize(*args)
 comp.setup('.')
 comp.initialize('frostnumber_model.cfg')
 comp.get_start_time()
 comp.get_end_time()
 comp.get_current_time()
 comp.update()
 comp.finalize()
===CMIP===
Download and install the `cmip` software from source.
 cd $install_dir/opt
 git clone https://github.com/permamodel/cmip
 cd cmip
 python setup.py develop
Babelize the CMIP tool.
 cd $install_dir
 mkdir -p build && cd build
 bmi-babelize $install_dir/opt/cmip --prefix=$install_dir/conda
Download the sample config file
and test the babelized CMIP component by starting a Python session in $install_dir/test
and executing the setup and IRF methods.
 from pymt.components import CMIP
 
 comp = CMIP()
 comp.get_component_name()
 comp.initialize('sample_cmip-siwenna.cfg')
 comp.get_start_time()
 comp.get_end_time()
 comp.get_current_time()
 comp.update()
 comp.get_current_time()
 comp.update_until(comp.get_end_time())
 comp.get_current_time()
 comp.finalize()