Package: coupledmodeldriver
CoupledModelDriverCoupledModelDriver generates an overlying job submission framework and configuration directories for NEMS-coupled coastal ocean
model ensembles.pipinstallcoupledmodeldriverIt utilizesNEMSpyto generate NEMS configuration files, shares common configurations between
runs, and organizes spinup and mesh partition into separate jobs for dependant submission.Documentation can be found athttps://coupledmodeldriver.readthedocs.iosupported models and platformsmodelscirculation modelsADCIRC (usesADCIRCpy)SCHISM (usesPySCHISM)forcingsATMESHWW3DATAHURDAT best trackOWIplatformslocalSlurmHeraStampede2Orionorganization / responsibilityCoupledModelDriver is developed for the COASTAL Act project by theCoastal Marine Modeling Branch (CMMB)of the Office of Coast Survey (OCS), a part of theNational Oceanic and Atmospheric Administration (NOAA), an agency of the United States federal government.Zachary Burnett (lead) -zachary.burnett@noaa.govWilliam Pringle -wpringle@anl.govSaeed Moghimi -saeed.moghimi@noaa.govusage example1. generate JSON configuration filesinitialize_adcirccreates JSON configuration files according to the given parameters. ADCIRC run options that are not exposed
by this command, such asrunsorgwce_solution_scheme, can be specified by directly modifying the JSON files. The following
creates JSON files for coupling(ATMESH + WW3DATA) -> ADCIRCover a small Shinnecock Inlet mesh:initialize_adcirc\--platformHERA\--mesh-directory/scratch2/COASTAL/coastal/save/shared/models/meshes/shinnecock/v1.0\--output-directoryhera_shinnecock_ike_spinup_tidal_atmesh_ww3data\--modeled-start-time20080823\--modeled-duration14:06:00:00\--modeled-timestep00:00:02\--nems-interval01:00:00\--adcirc-executable/scratch2/COASTAL/coastal/save/shared/repositories/CoastalApp/ALLBIN_INSTALL/NEMS-adcirc-atmesh-ww3data.x\--adcirc-processors40--adcprep-executable/scratch2/COASTAL/coastal/save/shared/repositories/CoastalApp/ALLBIN_INSTALL/adcprep\--modulefile/scratch2/COASTAL/coastal/save/shared/repositories/CoastalApp/modulefiles/envmodules_intel.hera\--forcingstidal,atmesh,ww3data\--tidal-sourceTPXO\--tidal-path/scratch2/COASTAL/coastal/save/shared/models/forcings/tides/h_tpxo9.v1.nc\--tidal-spinup-duration12:06:00:00\--atmesh-path/scratch2/COASTAL/coastal/save/shared/models/forcings/shinnecock/ike/wind_atm_fin_ch_time_vec.nc\--ww3data-path/scratch2/COASTAL/coastal/save/shared/models/forcings/shinnecock/ike/ww3.Constant.20151214_sxy_ike_date.ncThis will create the directoryhera_shinnecock_ike_spinup_tidal_atmesh_ww3data/with the following JSON configuration files:📂 hera_shinnecock_ike_spinup_tidal_atmesh_ww3data/
┣ 📜 configure_adcirc.json
┣ 📜 configure_atmesh.json
┣ 📜 configure_modeldriver.json
┣ 📜 configure_nems.json
┣ 📜 configure_slurm.json
┣ 📜 configure_tidal_forcing.json
┗ 📜 configure_ww3data.jsonThese files contain relevant configuration values for an ADCIRC run. You will likely wish to change these values to alter the
resulting run, before generating the actual model configuration. For instance, NEMS connections and the run sequence need to be
manually specified inconfigure_nems.json.2. generate model configuration filesgenerate_adcircgenerates an ADCIRC run configuration (fort.14,fort.15, etc.) using options read from the JSON
configuration files (generated in the previous step).cdhera_shinnecock_ike_spinup_tidal_atmesh_ww3data
generate_adcircThe resulting configuration will look like this:📂 hera_shinnecock_ike_spinup_tidal_atmesh_ww3data/
┣ 📜 configure_adcirc.json
┣ 📜 configure_atmesh.json
┣ 📜 configure_modeldriver.json
┣ 📜 configure_nems.json
┣ 📜 configure_slurm.json
┣ 📜 configure_tidal_forcing.json
┣ 📜 configure_ww3data.json
┣ 📂 spinup/
┃  ┣ 📜 fort.13
┃  ┣ 🔗 fort.14 -> ../fort.14
┃  ┣ 📜 fort.15
┃  ┣ 📜 nems.configure
┃  ┣ 📜 model_configure
┃  ┣ 🔗 atm_namelist.rc -> ./model_configure
┃  ┣ 📜 config.rc
┃  ┣ 📜 setup.job
┃  ┗ 📜 adcirc.job
┣ 📂 runs/
┃  ┗ 📂 unperturbed/
┃    ┣ 📜 fort.13
┃    ┣ 🔗 fort.14 -> ../../fort.14
┃    ┣ 📜 fort.15
┃    ┣ 🔗 fort.67.nc -> ../../spinup/fort.67.nc
┃    ┣ 🔗 fort.68.nc -> ../../spinup/fort.68.nc
┃    ┣ 📜 nems.configure
┃    ┣ 📜 model_configure
┃    ┣ 🔗 atm_namelist.rc -> ./model_configure
┃    ┣ 📜 config.rc
┃    ┣ 📜 setup.job
┃    ┗ 📜 adcirc.job
┣ 📜 fort.14
┣ 📜 cleanup.sh
┗ 📜 run_hera.sh3. run the modelThe previous step will also have generated a script called./run_hera.sh. You can run it to submit the model run to the Slurm
job queue:./run_hera.shThe queue will have the following jobs added:JOBID CPU NODE DEPENDENCY       NODELIST(REA NAME
20967647 1   1    (null)           (None)       ADCIRC_SETUP_SPINUP
20967648 40  1    afterok:20967647 (Dependency) ADCIRC_COLDSTART_SPINUP
20967649 1   1    (null)           (None)       ADCIRC_SETUP_unperturbed
20967650 42  2    afterok:20967649 (Dependency) ADCIRC_HOTSTART_unperturbed4. track model progresscheck_completionchecks the completion status of a running model directory.cdhera_shinnecock_ike_spinup_tidal_atmesh_ww3data
check_completion{"hera_shinnecock_ike_spinup_tidal_atmesh_ww3data":{"spinup":"running - 15%","runs":"not_started - 0%"}}you can also pass a specific directory (or several directories):check_completionspinup{"spinup":"running - 27%"}cdrun_20211027_florence_besttrack_250msubset_quadrature
check_completionruns/*_13{"vortex_4_variable_perturbation_13":"completed - 100.0%","vortex_4_variable_quadrature_13":"not_started - 0%"}if a run has an error, you can pass--verboseto see detailed logs:check_completionspinup{"spinup":"error - 0%"}check_completionspinup--verbose{"spinup":{"status":"error","progress":"0%","error":{"ADCIRC_SETUP_SPINUP.err.log":["forrtl: severe (24): end-of-file during read, unit -4, file /proc/92195/fd/0\n","Image              PC                Routine            Line        Source             \n","adcprep            000000000069A72E  Unknown               Unknown  Unknown\n","adcprep            00000000006CBAAF  Unknown               Unknown  Unknown\n","adcprep            000000000050A5CB  openprepfiles_           6996  prep.F\n","adcprep            0000000000507F22  prep13_                   753  prep.F\n","adcprep            000000000042E2E9  prepinput_                717  adcprep.F\n","adcprep            000000000042BCDB  MAIN__                    239  adcprep.F\n","adcprep            000000000040B65E  Unknown               Unknown  Unknown\n","libc-2.17.so       00002AAEC02EB555  __libc_start_main     Unknown  Unknown\n","adcprep            000000000040B569  Unknown               Unknown  Unknown\n","srun: error: h24c51: task 0: Exited with exit code 24\n","srun: launch/slurm: _step_signal: Terminating StepId=25366266.1\n"]}}}check_completionruns{"spinup":"failed - 0%"}check_completionruns--verbose{"runs":{"status":"failed","progress":"0%","failed":{"fort.16":"ADCIRC output file `fort.16` not found"},"error":{"ADCIRC_SETUP_unperturbed.err.log":["slurmstepd: error: execve(): /scratch2/COASTAL/coastal/save/shared/repositories/CoastalApp/ADCIRC/ALLBIN_INSTALL/adcprep: No such file or directory\n","srun: error: h18c49: task 0: Exited with exit code 2\n","srun: launch/slurm: _step_signal: Terminating StepId=25366268.0\n"]}}}