Repository: ESMCI/cime

Issue:
Exception triggered in NLCOMP
I updated the namelist in several pop tests (managing POP's history file output is complicated, and there were several different ways of dealing with that in the `testmods` directory which I would like to standardize). Anyway, all the tests return `NLFAIL` which is good, because I expected the namelist comparison to fail, but instead of seeing details about adding some variables and changing the value of others, I was getting the following

```
2017-11-30 14:37:40: Exception during namelist operations:
sequence item 0: expected string, tuple found
Traceback (most recent call last):
  File "/home/mlevy/codes/CESM/cesm2_0_beta07+marbl_dev/cime/scripts/Tools/../../scripts/lib/CIME/case_cmpgen_namelists.py", line 121, in case_cmpgen_namelists
    success, output = _do_full_nl_comp(case, test_name, compare_name, baseline_root)
  File "/home/mlevy/codes/CESM/cesm2_0_beta07+marbl_dev/cime/scripts/Tools/../../scripts/lib/CIME/case_cmpgen_namelists.py", line 45, in _do_full_nl_comp
    success, current_comments = compare_namelist_files(baseline_counterpart, item, test)
  File "/home/mlevy/codes/CESM/cesm2_0_beta07+marbl_dev/cime/scripts/Tools/../../scripts/lib/CIME/compare_namelists.py", line 540, in compare_namelist_files
    comments = _compare_namelists(gold_namelists, comp_namelists, case)
  File "/home/mlevy/codes/CESM/cesm2_0_beta07+marbl_dev/cime/scripts/Tools/../../scripts/lib/CIME/compare_namelists.py", line 524, in _compare_namelists
    comments += "".join(nlcomment)
TypeError: sequence item 0: expected string, tuple found
```

I haven't really had luck tracing this, as it seems like both `comments` and `nlcomment` are strings in the block

```
    comments = ""
    for namelist, nlcomment in different_namelists.iteritems():
        if len(nlcomment) == 1:
            comments += nlcomment[0]
        else:
            comments += "Differences in namelist '{}':\n".format(namelist)
            comments += "".join(nlcomment)
```

(above comes from `compare_namelists.py` and line 524 is the last line).

This is using `cime5.4.0-alpha.03` and the full difference of my changes to `user_nl_pop` are

```
$ svn diff cime_config/testdefs/testmods_dirs/pop/default/user_nl_pop
Index: cime_config/testdefs/testmods_dirs/pop/default/user_nl_pop
===================================================================
--- cime_config/testdefs/testmods_dirs/pop/default/user_nl_pop	(revision 87661)
+++ cime_config/testdefs/testmods_dirs/pop/default/user_nl_pop	(working copy)
@@ -1,7 +1,9 @@
 & tavg_nml
+    tavg_freq(1) = 1
+    tavg_file_freq(1) = 1
+    tavg_freq_opt(1) = 'nday'
+    tavg_file_freq_opt(1) = 'nday'
+    n_tavg_streams = 1
     ltavg_ignore_extra_streams = .true.
-    n_tavg_streams = 1
-    tavg_freq_opt = 'nday'
-    tavg_file_freq_opt = 'nday'
 /
 ldebug = .true.
```


Discussion:
@mlevy - I've had this problem before when changing pop's namelist. I can
help with this tomorrow.

On Thu, Nov 30, 2017 at 3:13 PM, Michael Levy <notifications@github.com>
wrote:

> I updated the namelist in several pop tests (managing POP's history file
> output is complicated, and there were several different ways of dealing
> with that in the testmods directory which I would like to standardize).
> Anyway, all the tests return NLFAIL which is good, because I expected the
> namelist comparison to fail, but instead of seeing details about adding
> some variables and changing the value of others, I was getting the following
>
> 2017-11-30 14:37:40: Exception during namelist operations:
> sequence item 0: expected string, tuple found
> Traceback (most recent call last):
>   File "/home/mlevy/codes/CESM/cesm2_0_beta07+marbl_dev/cime/scripts/Tools/../../scripts/lib/CIME/case_cmpgen_namelists.py", line 121, in case_cmpgen_namelists
>     success, output = _do_full_nl_comp(case, test_name, compare_name, baseline_root)
>   File "/home/mlevy/codes/CESM/cesm2_0_beta07+marbl_dev/cime/scripts/Tools/../../scripts/lib/CIME/case_cmpgen_namelists.py", line 45, in _do_full_nl_comp
>     success, current_comments = compare_namelist_files(baseline_counterpart, item, test)
>   File "/home/mlevy/codes/CESM/cesm2_0_beta07+marbl_dev/cime/scripts/Tools/../../scripts/lib/CIME/compare_namelists.py", line 540, in compare_namelist_files
>     comments = _compare_namelists(gold_namelists, comp_namelists, case)
>   File "/home/mlevy/codes/CESM/cesm2_0_beta07+marbl_dev/cime/scripts/Tools/../../scripts/lib/CIME/compare_namelists.py", line 524, in _compare_namelists
>     comments += "".join(nlcomment)
> TypeError: sequence item 0: expected string, tuple found
>
> I haven't really had luck tracing this, as it seems like both comments
> and nlcomment are strings in the block
>
>     comments = ""
>     for namelist, nlcomment in different_namelists.iteritems():
>         if len(nlcomment) == 1:
>             comments += nlcomment[0]
>         else:
>             comments += "Differences in namelist '{}':\n".format(namelist)
>             comments += "".join(nlcomment)
>
> (above comes from compare_namelists.py and line 524 is the last line).
>
> This is using cime5.4.0-alpha.03 and the full difference of my changes to
> user_nl_pop are
>
> $ svn diff cime_config/testdefs/testmods_dirs/pop/default/user_nl_pop
> Index: cime_config/testdefs/testmods_dirs/pop/default/user_nl_pop
> ===================================================================
> --- cime_config/testdefs/testmods_dirs/pop/default/user_nl_pop	(revision 87661)
> +++ cime_config/testdefs/testmods_dirs/pop/default/user_nl_pop	(working copy)
> @@ -1,7 +1,9 @@
>  & tavg_nml
> +    tavg_freq(1) = 1
> +    tavg_file_freq(1) = 1
> +    tavg_freq_opt(1) = 'nday'
> +    tavg_file_freq_opt(1) = 'nday'
> +    n_tavg_streams = 1
>      ltavg_ignore_extra_streams = .true.
> -    n_tavg_streams = 1
> -    tavg_freq_opt = 'nday'
> -    tavg_file_freq_opt = 'nday'
>  /
>  ldebug = .true.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/ESMCI/cime/issues/2103>, or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AHlxE6OQtziejOupPiI5m3ol7cLw5tEXks5s7yhugaJpZM4QxSA4>
> .
>


Resolving patch:
diff --git a/scripts/lib/CIME/compare_namelists.py b/scripts/lib/CIME/compare_namelists.py
--- a/scripts/lib/CIME/compare_namelists.py
+++ b/scripts/lib/CIME/compare_namelists.py
@@ -337,7 +337,7 @@ def _compare_values(name, gold_value, comp_value, case):
     comments = ""
     if (type(gold_value) != type(comp_value)):
         comments += "  variable '{}' did not have expected type '{}', instead is type '{}'\n".format(name, type(gold_value), type(comp_value))
-        return (False, comments)
+        return comments
 
     if (type(gold_value) is list):
         # Note, list values remain order sensitive


Test patch: