HydrologyExecutorBlanca

Instructions for installing and configuring a WMT executor on blanca.
--Mpiper (talk) 10:26, 1 October 2018 (MDT)
==Set build environment on blanca==
Login to summit.
 ssh mapi8461@login.rc.colorado.edu
Load the correct slurm for blanca.
 module load slurm/blanca
Login to a compute node.
 sinteractive
Build everything on the compute node.
==Set install and conda directories==
The install directory for this executor is /work/csdms/wmt/hydrology.
 install_dir=/work/csdms/wmt/hydrology
 mkdir -p $install_dir
Make sure read and execute bits are set on this directory. 
 chmod 0775 $install_dir
An Anaconda distribution is located at /work/csdms/anaconda3.
This distribution is used by all WMT executors on blanca.
Store this location and add it to your path.
 conda_dir=/work/csdms/anaconda3
 PATH=$conda_dir/bin:$PATH
WMT executors are installed as environments within this distribution.
==Install the CSDMS software stack==
Load blanca's `git` module.
 module load git
Clone the `wmt-executor-config-files` repo to a common location.
 cd $install_dir/..
 git clone https://github.com/csdms/wmt-executor-config-files
 cd wmt-executor-config-files
Install the base of the CSDMS software stack in a new environment.
 conda env create -f wmt-hydrology.yml
The stack will be installed in $conda_dir/envs/wmt-hydrology.
Note that there's no PyMT: this executor predates it.
There are install instructions below for the `coupling` environment.
===Install executor software===
Install the `wmt-exe` package from source.
 mkdir -p $install_dir/opt && cd $install_dir/opt
 git clone https://github.com/csdms/wmt-exe
 cd wmt-exe
 python setup.py develop
Create a site configuration file that describes the executor and symlink it to the executor's etc/ directory.
 work_dir="/rc_scratch/$USER/wmt/hydrology"
 python setup.py configure --wmt-prefix=$install_dir --launch-dir=$work_dir --exec-dir=$work_dir
 ln -s "$(realpath wmt.cfg)" $conda_dir/envs/wmt-hydrology/etc
Check that `$USER` didn't get expanded in the file. Lines 10-11 should be:
 exec_dir = /rc_scratch/$USER/wmt/hydrology
 launch_dir = /rc_scratch/$USER/wmt/hydrology
Note that we're using /rc_scratch for the launch and execution directories instead of the default ~/.wmt.
Also note that we needed an SbatchLauncher class
for wmt-exe because blanca uses Slurm
instead of Torque for job control.
==Install Topoflow==
The following directions rely on ancient technology,
but they work!
Start by installing the coupling framework and the componentized topoflow package.
 conda install coupling --override-channels -c csdms/channel/dev -c defaults
 conda install csdms-topoflow --override-channels -c csdms/channel/dev -c csdms -c defaults
Next, some trickery.
Uninstall `topoflow`, which also uninstalls `topoflow-bridge` and `csdms-topoflow`,
but doesn't uninstall the deps of `csdms-topoflow`, like `babel`.
Reinstall only `csdms-topoflow`, which is still needed.
 conda uninstall topoflow
 conda install csdms-topoflow --no-deps --override-channels -c csdms/channel/dev -c csdms -c defaults
Also uninstall the coupling framework.
 conda uninstall coupling
Then get the development version of the coupling framework and install it.
 mkdir -p $install_dir/opt && cd $install_dir/opt
 git clone https://github.com/csdms/coupling
 cd coupling/
 git checkout mcflugen/fix-for-new-bmi 
 python setup.py develop
Get the development versions of `topoflow` and `topoflow-bridge` and install them.
 cd ..
 git clone https://github.com/mdpiper/topoflow
 cd topoflow
 python setup.py develop
 cd ..
 git clone https://github.com/csdms/topoflow-bridge
 cd topoflow-bridge
 python setup.py develop
Manually copy files from the development version of `topoflow-bridge`
into the executor's version.
 src=$install_dir/opt/topoflow-bridge/.bmi
 dst=$conda_dir/envs/wmt-hydrology/share/csdms
 cp $src/channels_diffusive_wave/* $dst/ChannelsDiffusiveWave
 cp $src/channels_dynamic_wave/* $dst/ChannelsDynamicWave
 ...
That's it!