Repository: ESMCI/cime

Issue:
With testmods, default directory sometimes occurs after others
@mnlevy1981 and I were just discussing this, connected with his recent comment in #713 

In some cases where you have an include_user_mods with two or more includes, it's possible for the various directories to be parsed in a way such that the 'default' directory (i.e., the directory at the bottom of the includes tree) appears in the middle of the user_nl file, rather than at the start. We both feel it should appear at the start, so that any other testmods will override settings set by default.

Here's an example; there are a number of examples like this in the POP test suite:

```
          ciso_abio
          /       \
        abio     ciso
        /          \
    default     default
```

Here, what we'd want is for the 'default' to appear at the top of user_nl_pop. Instead, it appears that the parsing happens like this:
1. ciso_abio: user_nl_pop is written
2. abio: user_nl_pop is pre-pended
3. default: user_nl_pop is pre-pended
4. ciso: user_nl_pop is pre-pended
5. (default is ignored)

So the resulting user_nl_pop would look like:

```
ciso
default
abio
ciso_abio
```

whereas we want

```
default
ciso
abio
ciso_abio
```

@mnlevy1981 's suggestion for how to accomplish this is to first build up a list of include directories. The key point is: whenever you encounter a duplicate, remove the existing instance, just keeping it at the end.

So in this case, the list would get built up like this:
1. abio
2. abio, default
3. abio, default, ciso
4. abio, ~~default,~~ ciso, default

Then you run through the list using code like we already have, doing the necessary pre-pends, resulting in:
1. ciso_abio: user_nl_pop is written
2. abio: user_nl_pop is pre-pended
3. ciso: user_nl_pop is pre-pended
4. default: user_nl_pop is pre-pended

resulting in the desired user_nl_pop.

Note also that the check for existing contents could then be removed.



Discussion:

Resolving patch:
diff --git a/utils/python/CIME/user_mod_support.py b/utils/python/CIME/user_mod_support.py
--- a/utils/python/CIME/user_mod_support.py
+++ b/utils/python/CIME/user_mod_support.py
@@ -22,6 +22,7 @@ def apply_user_mods(caseroot, user_mods_path, ninst=None):
             os.remove(shell_command_file)
 
     include_dirs = build_include_dirs_list(user_mods_path)
+    logger.debug("include_dirs are %s"%include_dirs)
     for include_dir in include_dirs:
         # write user_nl_xxx file in caseroot
         for user_nl in glob.iglob(os.path.join(include_dir,"user_nl_*")):
@@ -75,7 +76,10 @@ def update_user_nl_file(case_user_nl, contents):
     if os.path.isfile(case_user_nl):
         with open(case_user_nl, "r") as fd:
             old_contents = fd.read()
-        if old_contents.find(contents) == -1:
+
+        oc = set(old_contents.splitlines())
+        nc = set(contents.splitlines())
+        if not nc.issubset(oc):
             contents = contents + old_contents
             update_file = True
         else:
@@ -101,9 +105,16 @@ def build_include_dirs_list(user_mods_path, include_dirs=None):
            "Expected full directory path, got '%s'"%user_mods_path)
     expect(os.path.isdir(user_mods_path),
            "Directory not found %s"%user_mods_path)
-    logger.info("Adding user mods directory %s"%user_mods_path)
-    include_dirs.append(os.path.normpath(user_mods_path))
-    include_file = os.path.join(include_dirs[-1],"include_user_mods")
+    norm_path = os.path.normpath(user_mods_path)
+
+    for dir_ in include_dirs:
+        if norm_path == dir_:
+            include_dirs.remove(norm_path)
+            break
+
+    logger.info("Adding user mods directory %s"%norm_path)
+    include_dirs.append(norm_path)
+    include_file = os.path.join(norm_path,"include_user_mods")
     if os.path.isfile(include_file):
         with open(include_file, "r") as fd:
             for newpath in fd:
@@ -115,4 +126,5 @@ def build_include_dirs_list(user_mods_path, include_dirs=None):
                         build_include_dirs_list(newpath, include_dirs)
                     else:
                         logger.warn("Could not resolve path '%s' in file '%s'"%newpath,include_file)
+
     return include_dirs


Test patch: