Repository: ESMCI/cime

Issue:
Options to case.build to speed incremental rebuilds
I feel like this was discussed earlier in one or more issues, but I can't find them, so I'm opening a new issue.

When I'm building new code that I've written, I usually have a lot of compilation errors initially (I know, it's probably just me...), and so I need to incrementally rebuild (rerunning ./case.build) a few times before the build is successful.

This currently takes quite a bit longer than it needs to, because before ./case.build gets to the library in question, it:

(a) Builds component namelists – sometimes twice, at least for some test types

(b) Checks whether all preceding libraries need to be rebuilt

These incremental rebuilds could be sped up if we had options to case.build to:

(1) skip namelist generation (addressing (a))

(2) build only a single, given library (addressing (b))... maybe this would imply also skipping namelist generation

What do people think about adding these or other options to speed up incremental rebuilds?


Discussion:
@billsacks have you tried using the --model-only flag to case.build?
I agree that a --skip-buildnml option would be nice, as @jgfouca points out --model-only will skip the shared libraries.  
I think it should skip the buildnml too since that's only done for the shared libs.
I just tried it, it does not skip buildnml.
```
 ./case.build --model-only
Building test for SMS in directory /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/scripts/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p
WARNING: Found difference in test REST_OPTION: case: none original value $STOP_OPTION
Resetting case due to detected re-run of phase MODEL_BUILD
Successfully cleaned batch script case.run
Creating batch script case.run
Writing case.run script from input template /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/config/cesm/machines/template.case.run
Writing case.st_archive script from input template /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/config/cesm/machines/template.st_archive
Locking file env_mach_pes.xml
If an old case build already exists, might want to run 'case.build --clean' before building
Generating component namelists as part of setup
Creating component namelists
   Calling /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/src/components/data_comps/datm/cime_config/buildnml
   Calling /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/src/components/stub_comps/slnd/cime_config/buildnml
   Calling /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/src/components/stub_comps/sice/cime_config/buildnml
   Calling /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/src/components/data_comps/docn/cime_config/buildnml
   Calling /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/src/components/stub_comps/srof/cime_config/buildnml
   Calling /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/src/components/stub_comps/sglc/cime_config/buildnml
   Calling /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/src/components/stub_comps/swav/cime_config/buildnml
   Calling /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/src/components/data_comps/desp/cime_config/buildnml
   Calling /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/src/drivers/mct/cime_config/buildnml
Finished creating component namelists
Building case in directory /glade/p/work/jedwards/sandboxes/cesm2_0_alpha/cime.jet/scripts/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p
sharedlib_only is False
model_only is True
Building atm with output to /glade/scratch/jedwards/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p/bld/atm.bldlog.170817-135728
Building lnd with output to /glade/scratch/jedwards/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p/bld/lnd.bldlog.170817-135728
Building ice with output to /glade/scratch/jedwards/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p/bld/ice.bldlog.170817-135728
Building ocn with output to /glade/scratch/jedwards/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p/bld/ocn.bldlog.170817-135728
Building rof with output to /glade/scratch/jedwards/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p/bld/rof.bldlog.170817-135728
Building glc with output to /glade/scratch/jedwards/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p/bld/glc.bldlog.170817-135728
Building wav with output to /glade/scratch/jedwards/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p/bld/wav.bldlog.170817-135728
Building esp with output to /glade/scratch/jedwards/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p/bld/esp.bldlog.170817-135728
sice built in 1.820498 seconds
srof built in 1.820566 seconds
desp built in 1.787297 seconds
slnd built in 1.832907 seconds
swav built in 1.821898 seconds
docn built in 1.837395 seconds
sglc built in 1.838389 seconds
datm built in 1.844381 seconds
Building cesm with output to /glade/scratch/jedwards/SMS_C3.f19_f19.ADESP.cheyenne_intel.20170816_151727_u2c70p/bld/cesm.bldlog.170817-135728 
Time spent not building: 0.502211 sec
Time spent building: 17.161353 sec
```
@jedwards4b shoot. I was looking at this code here:

```
    if not model_only:
        logger.info("Generating component namelists as part of build")
        create_namelists(case)
```
It looks like your case.build called case.setup?
oh - good point, I think because it was a test?
```
% ./create_test cime_tiny --no-build
...
% cd to one of the cases
% ./case.build --sharedlib-only
...
Generating component namelists as part of build
Creating component namelists
...
% ./case.build --model-only
...
Building acme, no buildnmls called
```

So --model-only appears to do exactly what @billsacks wants when I use it.
Yes, I think so.   @billsacks do you agree?
I most often am changing files that are part of the sharedlib build - either CLM or cime code - so, no, `--model-only` won't help me.
Can someone remind me why a sharedlib build would ever need a round of buildnml in the first place?
another suggestion might be individual component options matching what we use for clean...
for example: case.build lnd 

> another suggestion might be individual component options matching what we use for clean...
> for example: case.build lnd

This would be very helpful.

Resolving patch:
diff --git a/scripts/lib/CIME/build.py b/scripts/lib/CIME/build.py
--- a/scripts/lib/CIME/build.py
+++ b/scripts/lib/CIME/build.py
@@ -12,12 +12,14 @@
 
 ###############################################################################
 def _build_model(build_threaded, exeroot, clm_config_opts, incroot, complist,
-                 lid, caseroot, cimeroot, compiler):
+                 lid, caseroot, cimeroot, compiler, buildlist):
 ###############################################################################
     logs = []
 
     thread_bad_results = []
     for model, comp, nthrds, _, config_dir in complist:
+        if buildlist is not None and model.lower() not in buildlist:
+            continue
 
         # aquap has a dependency on atm so we will build it after the threaded loop
         if comp == "aquap":
@@ -73,34 +75,35 @@ def _build_model(build_threaded, exeroot, clm_config_opts, incroot, complist,
     # Now build the executable
     #
 
-    cime_model = get_model()
-    file_build = os.path.join(exeroot, "{}.bldlog.{}".format(cime_model, lid))
-
-    config_dir = os.path.join(cimeroot, "src", "drivers", "mct", "cime_config")
-    f = open(file_build, "w")
-    bldroot = os.path.join(exeroot, "cpl", "obj")
-    if not os.path.isdir(bldroot):
-        os.makedirs(bldroot)
-    logger.info("Building {} with output to {} ".format(cime_model, file_build))
-    stat = run_cmd("{}/buildexe {} {} {}"
-                   .format(config_dir, caseroot, libroot, bldroot),
-                   from_dir=bldroot, verbose=False, arg_stdout=f,
-                   arg_stderr=subprocess.STDOUT)[0]
-    f.close()
-    analyze_build_log("{} exe".format(cime_model), file_build, compiler)
-    expect(stat == 0, "BUILD FAIL: buildexe failed, cat {}".format(file_build))
+    if not buildlist:
+        cime_model = get_model()
+        file_build = os.path.join(exeroot, "{}.bldlog.{}".format(cime_model, lid))
+
+        config_dir = os.path.join(cimeroot, "src", "drivers", "mct", "cime_config")
+        f = open(file_build, "w")
+        bldroot = os.path.join(exeroot, "cpl", "obj")
+        if not os.path.isdir(bldroot):
+            os.makedirs(bldroot)
+        logger.info("Building {} with output to {} ".format(cime_model, file_build))
+        stat = run_cmd("{}/buildexe {} {} {}"
+                       .format(config_dir, caseroot, libroot, bldroot),
+                       from_dir=bldroot, verbose=False, arg_stdout=f,
+                       arg_stderr=subprocess.STDOUT)[0]
+        f.close()
+        analyze_build_log("{} exe".format(cime_model), file_build, compiler)
+        expect(stat == 0, "BUILD FAIL: buildexe failed, cat {}".format(file_build))
 
-    # Copy the just-built ${MODEL}.exe to ${MODEL}.exe.$LID
-    shutil.copy("{}/{}.exe".format(exeroot, cime_model), "{}/{}.exe.{}".format(exeroot, cime_model, lid))
+        # Copy the just-built ${MODEL}.exe to ${MODEL}.exe.$LID
+        shutil.copy("{}/{}.exe".format(exeroot, cime_model), "{}/{}.exe.{}".format(exeroot, cime_model, lid))
 
-    logs.append(file_build)
+        logs.append(file_build)
 
     return logs
 
 ###############################################################################
 def _build_checks(case, build_threaded, comp_interface, use_esmf_lib,
                   debug, compiler, mpilib, complist, ninst_build, smp_value,
-                  model_only):
+                  model_only, buildlist):
 ###############################################################################
     """
     check if a build needs to be done and warn if a clean is warrented first
@@ -196,14 +199,14 @@ def _build_checks(case, build_threaded, comp_interface, use_esmf_lib,
     create_dirs(case)
 
     case.flush()
-    if not model_only:
+    if not model_only and not buildlist:
         logger.info("Generating component namelists as part of build")
         create_namelists(case)
 
     return sharedpath
 
 ###############################################################################
-def _build_libraries(case, exeroot, sharedpath, caseroot, cimeroot, libroot, lid, compiler):
+def _build_libraries(case, exeroot, sharedpath, caseroot, cimeroot, libroot, lid, compiler, buildlist):
 ###############################################################################
 
     shared_lib = os.path.join(exeroot, sharedpath, "lib")
@@ -218,6 +221,9 @@ def _build_libraries(case, exeroot, sharedpath, caseroot, cimeroot, libroot, lid
     logs = []
     sharedlibroot = os.path.abspath(case.get_value("SHAREDLIBROOT"))
     for lib in libs:
+        if buildlist is not None and lib not in buildlist:
+            continue
+
         if lib == "csm_share":
             # csm_share adds its own dir name
             full_lib_path = os.path.join(sharedlibroot, sharedpath)
@@ -248,12 +254,11 @@ def _build_libraries(case, exeroot, sharedpath, caseroot, cimeroot, libroot, lid
                 if re.search("Current setting for", line):
                     logger.warn(line)
 
-
     # clm not a shared lib for ACME
-    if get_model() != "acme":
+    if get_model() != "acme" and (buildlist is None or "lnd" in buildlist):
         comp_lnd = case.get_value("COMP_LND")
         clm_config_opts = case.get_value("CLM_CONFIG_OPTS")
-        if comp_lnd == "clm" and not "clm4_0" in clm_config_opts:
+        if comp_lnd == "clm" and "clm4_0" not in clm_config_opts:
             logging.info("         - Building clm4_5/clm5_0 Library ")
             esmfdir = "esmf" if case.get_value("USE_ESMF_LIB") else "noesmf"
             bldroot = os.path.join(sharedlibroot, sharedpath, case.get_value("COMP_INTERFACE"), esmfdir, "clm","obj" )
@@ -356,7 +361,7 @@ def _clean_impl(case, cleanlist, clean_all):
     case.flush()
 
 ###############################################################################
-def _case_build_impl(caseroot, case, sharedlib_only, model_only):
+def _case_build_impl(caseroot, case, sharedlib_only, model_only, buildlist):
 ###############################################################################
 
     t1 = time.time()
@@ -493,23 +498,24 @@ def _case_build_impl(caseroot, case, sharedlib_only, model_only):
 
     sharedpath = _build_checks(case, build_threaded, comp_interface,
                                use_esmf_lib, debug, compiler, mpilib,
-                               complist, ninst_build, smp_value, model_only)
+                               complist, ninst_build, smp_value, model_only, buildlist)
 
     t2 = time.time()
     logs = []
 
     if not model_only:
         logs = _build_libraries(case, exeroot, sharedpath, caseroot,
-                               cimeroot, libroot, lid, compiler)
+                               cimeroot, libroot, lid, compiler, buildlist)
 
     if not sharedlib_only:
         os.environ["INSTALL_SHAREDPATH"] = os.path.join(exeroot, sharedpath) # for MPAS makefile generators
         logs.extend(_build_model(build_threaded, exeroot, clm_config_opts, incroot, complist,
-                                lid, caseroot, cimeroot, compiler))
+                                lid, caseroot, cimeroot, compiler, buildlist))
 
-    if not sharedlib_only:
+    if not sharedlib_only and not buildlist:
         # in case component build scripts updated the xml files, update the case object
         case.read_xml()
+        # Note, doing buildlists will never result in the system thinking the build is complete
         post_build(case, logs)
 
     t3 = time.time()
@@ -553,9 +559,9 @@ def post_build(case, logs):
     save_build_provenance(case, lid=lid)
 
 ###############################################################################
-def case_build(caseroot, case, sharedlib_only=False, model_only=False):
+def case_build(caseroot, case, sharedlib_only=False, model_only=False, buildlist=None):
 ###############################################################################
-    functor = lambda: _case_build_impl(caseroot, case, sharedlib_only, model_only)
+    functor = lambda: _case_build_impl(caseroot, case, sharedlib_only, model_only, buildlist)
     return run_and_log_case_status(functor, "case.build", caseroot=caseroot)
 
 ###############################################################################


Test patch: