Repository: ESMCI/cime

Issue:
Enhancement to nmlgen: Allow setting defaults for specific namelist groups conditionally
(Preface: I know that we are not very invested in continued maintenance of nmlgen.py for CESM because of @alperaltuntas 's new infrastructure, but I think that what I'm proposing here will be pretty easy to implement, and will help with CISM's buildnml python rewrite.)

For CISM's build-namelist, there are a few namelist groups that we only add to the namelist if a certain other namelist variable has a certain setting. For example, we only add the isostasy namelist group if the isostasy namelist variable is set to "on". This conditional addition of namelist groups can be done already, but it's currently awkward to add error checking to confirm that the user hasn't tried to set any invalid items in user_nl_cism: for example, that they aren't trying to set the value for some isostasy-related variable when isostasy is off (so the setting they're trying to make would never appear in the generated namelist). (Currently, I think this can only be done by listing all variables in the group explicitly in buildnml, which goes against the data-driven design.)

I think I can support this error checking with a few small changes to nmlgen.py and namelist_definition.py; I plan to do this in the next day or so unless anyone tells me that it's a terrible idea :-)

- Add a new optional argument to `init_defaults` (in nmlgen.py) and `set_nodes` (in namelist_definition.py): `skip_default_for_groups`. If provided, this should be a list of group names; any namelist variable in one of the given groups will _not_ be added to the list of `default_nodes`.

- Add a new function in `nmlgen.py`: `add_defaults_for_group(group)`. This will call `add_default` for all variables in the given group.

- Add a new function in `nmlgen.py`: `confirm_empty(group)`. This will ensure that no variables have had values set in the given group.

Then, in CISM, we can have code like this:

```python
nmlgen.init_defaults(infile, config, skip_default_for_groups=["isostasy", ...])

# ...

if int(nmlgen.get_value("isostasy")) == _ISOSTASY_ON:
    nmlgen.add_defaults_for_group("isostasy")
else:
    nmlgen.confirm_empty("isostasy")
```


Discussion:
This could be useful for CTSM where we have the same sort of thing going on. But, right now we do a lot of custom code checking to ensure this is true. This general check would be an improvement to what we currently do.
@ekluzek Thanks. Yes, I was thinking that, too, though I'm really hoping that we'll be able to leverage @alperaltuntas 's infrastructure for CTSM. (I have time pressure to get CISM's in, and it's much easier than CTSM's, so I'm going ahead with it before @alperaltuntas 's infrastructure has been incorporated.)

Resolving patch:
diff --git a/scripts/lib/CIME/XML/namelist_definition.py b/scripts/lib/CIME/XML/namelist_definition.py
--- a/scripts/lib/CIME/XML/namelist_definition.py
+++ b/scripts/lib/CIME/XML/namelist_definition.py
@@ -101,7 +101,7 @@ def set_nodes(self, skip_groups=None):
             per_stream_entry = self.get(node, "per_stream_entry") == "true"
             set_node_values = False
             if skip_groups:
-                group_name = self._get_group_name(node)
+                group_name = self.get_group_name(node)
                 if not group_name in skip_groups:
                     self._entry_nodes.append(node)
                     set_node_values = True
@@ -118,10 +118,10 @@ def set_nodes(self, skip_groups=None):
                 self._nodes[name] = node
                 self._entry_types[name] = self._get_type(node)
                 self._valid_values[name] = self._get_valid_values(node)
-                self._group_names[name] = self._get_group_name(node)
+                self._group_names[name] = self.get_group_name(node)
         return default_nodes
 
-    def _get_group_name(self, node=None):
+    def get_group_name(self, node=None):
         if self.get_version() == 1.0:
             group = self.get(node, 'group')
         elif self.get_version() >= 2.0:
diff --git a/scripts/lib/CIME/nmlgen.py b/scripts/lib/CIME/nmlgen.py
--- a/scripts/lib/CIME/nmlgen.py
+++ b/scripts/lib/CIME/nmlgen.py
@@ -90,6 +90,10 @@ def __init__(self, case, definition_files, files=None):
         # Create namelist object.
         self._namelist = Namelist()
 
+        # entries for which we should potentially call add_default (variables that do not
+        # set skip_default_entry)
+        self._default_nodes = []
+
     # Define __enter__ and __exit__ so that we can use this as a context manager
     def __enter__(self):
         return self
@@ -97,16 +101,25 @@ def __enter__(self):
     def __exit__(self, *_):
         return False
 
-    def init_defaults(self, infiles, config, skip_groups=None, skip_entry_loop=False):
+    def init_defaults(self, infiles, config, skip_groups=None, skip_entry_loop=False,
+                      skip_default_for_groups=None):
         """Return array of names of all definition nodes
+
+        If skip_default_for_groups is provided, it should be a list of namelist group
+        names; the add_default call will not be done for any variables in these
+        groups. This is often paired with later conditional calls to
+        add_defaults_for_group.
         """
+        if skip_default_for_groups is None:
+            skip_default_for_groups = []
+
         # first clean out any settings left over from previous calls
         self.new_instance()
 
         self._definition.set_nodes(skip_groups=skip_groups)
 
         # Determine the array of entry nodes that will be acted upon
-        entry_nodes = self._definition.set_nodes(skip_groups=skip_groups)
+        self._default_nodes = self._definition.set_nodes(skip_groups=skip_groups)
 
         # Add attributes to definition object
         self._definition.add_attributes(config)
@@ -128,10 +141,47 @@ def init_defaults(self, infiles, config, skip_groups=None, skip_entry_loop=False
             self._namelist.merge_nl(new_namelist)
 
         if not skip_entry_loop:
-            for entry in entry_nodes:
+            for entry in self._default_nodes:
+                group_name = self._definition.get_group_name(entry)
+                if not group_name in skip_default_for_groups:
+                    self.add_default(self._definition.get(entry, "id"))
+
+        return [self._definition.get(entry, "id") for entry in self._default_nodes]
+
+    def add_defaults_for_group(self, group):
+        """Call add_default for namelist variables in the given group
+
+        This still skips variables that have attributes of skip_default_entry or
+        per_stream_entry.
+
+        This must be called after init_defaults. It is often paired with use of
+        skip_default_for_groups in the init_defaults call.
+        """
+        for entry in self._default_nodes:
+            group_name = self._definition.get_group_name(entry)
+            if group_name == group:
                 self.add_default(self._definition.get(entry, "id"))
 
-        return [self._definition.get(entry, "id") for entry in entry_nodes]
+    def confirm_group_is_empty(self, group_name, errmsg):
+        """Confirms that no values have been added to the given group
+
+        If any values HAVE been added to this group, aborts with the given error message.
+
+        This is often paired with use of skip_default_for_groups in the init_defaults call
+        and add_defaults_for_group, as in:
+
+            if nmlgen.get_value("enable_frac_overrides") == ".true.":
+                nmlgen.add_defaults_for_group("glc_override_nml")
+            else:
+                nmlgen.confirm_empty("glc_override_nml", "some message")
+
+        Args:
+        group_name: string - name of namelist group
+        errmsg: string - error message to print if group is not empty
+        """
+        variables_in_group = self._namelist.get_variable_names(group_name)
+        fullmsg = "{}\nOffending variables: {}".format(errmsg, variables_in_group)
+        expect(len(variables_in_group) == 0, fullmsg)
 
     @staticmethod
     def quote_string(string):


Test patch: