internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ESMCI/cime

Issue:
All component builds mislabeled as 'clm builds'
All component builds are labeled as clm builds. 
```
Component csm_share build complete with 3 warnings
         - Building clm Library 
Building atm with output to /homes/jayesh/acme/scratch/test_X_f19_g16/bld/atm.bldlog.170309-084354
         - Building clm Library 
Building lnd with output to /homes/jayesh/acme/scratch/test_X_f19_g16/bld/lnd.bldlog.170309-084354
...
```
I tried it on anlworkstation with master (cime5.3.0-alpha.02-9-g5f4f989) but should be an issue on other systems too.


Discussion:
Looks like the bug is in build.py@line:41 at
```
        else:
            logger.info("         - Building clm Library ")


```
This patch should fix the problem,
```
diff --git a/scripts/lib/CIME/build.py b/scripts/lib/CIME/build.py
index 3bbf2c1..6aa1be8 100644
--- a/scripts/lib/CIME/build.py
+++ b/scripts/lib/CIME/build.py
@@ -38,7 +38,7 @@ def _build_model(build_threaded, exeroot, clm_config_opts, incroot, comp
                 else:
                     continue
         else:
-            logger.info("         - Building clm Library ")
+            logger.info("         - Building %s Library ", model)
 
         smp = nthrds > 1 or build_threaded

```
@jgfouca : Is this the right fix? 
@jayeshkrishna , yes, putting it in now.
You could also put this in the cime5.2.0 branch in ACME.  Otherwise we'll have to live with it there for a while.

Resolving patch:
diff --git a/scripts/lib/CIME/build.py b/scripts/lib/CIME/build.py
--- a/scripts/lib/CIME/build.py
+++ b/scripts/lib/CIME/build.py
@@ -29,7 +29,7 @@ def _build_model(build_threaded, exeroot, clm_config_opts, incroot, complist,
             continue
 
         # special case for clm
-        # clm 4_0 is not a shared library and must be built here
+        # clm 4_0 is not a shared (as in sharedlibs, shared by all tests) library and must be built here
         # clm 4_5 and newer is a shared library (but not in ACME) and should be built in build_libraries
         if get_model() != "acme":
             if comp == "clm":
@@ -38,7 +38,7 @@ def _build_model(build_threaded, exeroot, clm_config_opts, incroot, complist,
                 else:
                     continue
         else:
-            logger.info("         - Building clm Library ")
+            logger.info("         - Building %s Library " % model)
 
         smp = nthrds > 1 or build_threaded
 


Test patch:
Repository: ESMCI/cime

Issue:
PR #1230 appears to break batch systems 
After merging #1230 I get an error from Z_FullSystemTest:

Traceback (most recent call last):
  File "./scripts_regression_tests.py", line 1175, in test_full_system
    self.assertTrue(test_time > 0, msg="test time was zero for %s" % test_status)
AssertionError: test time was zero for /scratch/cluster/jedwards/scripts_regression_test.20170313_145646/ERR.f45_g37_rx1.A.hobart_intel.fake_testing_only_20170313_151740/TestStatus


All of the tests actually passed.  


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/case_submit.py b/scripts/lib/CIME/case_submit.py
--- a/scripts/lib/CIME/case_submit.py
+++ b/scripts/lib/CIME/case_submit.py
@@ -71,9 +71,13 @@ def submit(case, job=None, resubmit=False, no_batch=False, batch_args=None):
         casebaseid = case.get_value("CASEBASEID")
         # This should take care of the race condition where the submitted job
         # begins immediately and tries to set RUN phase. We proactively assume
-        # a passed SUBMIT phase.
+        # a passed SUBMIT phase. If this state is already PASS, don't set it again
+        # because then we'll lose RUN phase info if it's there. This info is important
+        # for system_tests_common to know if it needs to reinitialize the test or not.
         with TestStatus(test_dir=caseroot, test_name=casebaseid) as ts:
-            ts.set_status(SUBMIT_PHASE, TEST_PASS_STATUS)
+            phase_status = ts.get_status(SUBMIT_PHASE)
+            if phase_status != TEST_PASS_STATUS:
+                ts.set_status(SUBMIT_PHASE, TEST_PASS_STATUS)
 
     try:
         functor = lambda: _submit(case, job, resubmit, no_batch, batch_args)


Test patch:
diff --git a/scripts/lib/CIME/test_scheduler.py b/scripts/lib/CIME/test_scheduler.py
--- a/scripts/lib/CIME/test_scheduler.py
+++ b/scripts/lib/CIME/test_scheduler.py
@@ -737,6 +737,8 @@ def _setup_cs_files(self):
             template = template.replace("<PATH>",
                                         os.path.join(self._cime_root,"scripts","Tools")).replace\
                                         ("<TESTID>", self._test_id)
+            if not os.path.exists(self._test_root):
+                os.makedirs(self._test_root)
             cs_status_file = os.path.join(self._test_root, "cs.status.%s" % self._test_id)
             with open(cs_status_file, "w") as fd:
                 fd.write(template)
Repository: ESMCI/cime

Issue:
Want more informative error message when you need to specify --user-compset
When you neglect to specify `--user-compset`, you get this message:

```
[yslogin4:/glade/p/work/sacks/cime/scripts]$ ./create_newcase -case test_a_user -compset 1850_DATM%NYF_SLND_DICE%SSMI_DOCN%DOM_DROF%NYF_SGLC_SWAV -res f45_g37_rx1
ERROR: Could not find a compset match for either alias or longname in 1850_DATM%NYF_SLND_DICE%SSMI_DOCN%DOM_DROF%NYF_SGLC_SWAV
```

It would be better if the message directed you to specify `--user-compset`


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/case.py b/scripts/lib/CIME/case.py
--- a/scripts/lib/CIME/case.py
+++ b/scripts/lib/CIME/case.py
@@ -447,7 +447,8 @@ def _set_compset_and_pesfile(self, compset_name, files, user_compset=False, pesf
             self.set_lookup_value("PES_SPEC_FILE", pesfile)
         else:
             expect(False,
-                   "Could not find a compset match for either alias or longname in %s" %(compset_name))
+                   "Could not find a compset match for either alias or longname in %s\n"%(compset_name)
+                   + "You may need the --user-compset argument.")
 
         return None, science_support
 


Test patch:
Repository: ESMCI/cime

Issue:
Batch error messages not passed to user.
"Submitting a second job to the debug queue on Titan (where only one is allowed) generates a " " error message. Before it passed through the system error message (that there was already a debug job in the queue)."

This is a regression from 5.1. to 5.2.


Discussion:
@rljacob when I use create_test, I see the error pretty clearly logged in TestStatus.log. Can you give me a bit more context? Is this just with case.submit?
Actually, it looks like case.submit produces an error as well, at least on skybridge when I intentionally sabotage the sbatch flags:

```
ERROR: Command: 'sbatch   --time 0:10:00 -p ec --account fy150001 case.test ' failed with error 'sbatch: unrecognized option '--fioasdasda=16'
sbatch: error: Try "sbatch --help" for more information'
```
The above is titan-specific so likely PBS.
And yes this is from executing case.submit by hand.
@rljacob I bet I know what the problem is. The titan batch command is not well behaved and is sending error output to stdout.

Resolving patch:
diff --git a/scripts/lib/CIME/XML/env_batch.py b/scripts/lib/CIME/XML/env_batch.py
--- a/scripts/lib/CIME/XML/env_batch.py
+++ b/scripts/lib/CIME/XML/env_batch.py
@@ -384,7 +384,7 @@ def submit_single_job(self, case, job, depid=None, no_batch=False, batch_args=No
                 submitcmd += string + " "
 
         logger.info("Submitting job script %s"%submitcmd)
-        output = run_cmd_no_fail(submitcmd)
+        output = run_cmd_no_fail(submitcmd + " 2>&1")
         jobid = self.get_job_id(output)
         logger.info("Submitted job id is %s"%jobid)
         return jobid
diff --git a/scripts/lib/CIME/utils.py b/scripts/lib/CIME/utils.py
--- a/scripts/lib/CIME/utils.py
+++ b/scripts/lib/CIME/utils.py
@@ -209,8 +209,12 @@ def run_cmd_no_fail(cmd, input_str=None, from_dir=None, verbose=None,
     'foo'
     """
     stat, output, errput = run_cmd(cmd, input_str, from_dir, verbose, arg_stdout, arg_stderr)
-    expect(stat == 0, "Command: '%s' failed with error '%s'%s" %
-           (cmd, errput, "" if from_dir is None else " from dir '%s'" % from_dir))
+    if stat != 0:
+        # If command produced no errput, put output in the exception since we
+        # have nothing else to go on.
+        errput = output if errput == "" else errput
+        expect(False, "Command: '%s' failed with error '%s'%s" %
+               (cmd, errput, "" if from_dir is None else " from dir '%s'" % from_dir))
     return output
 
 def check_minimum_python_version(major, minor):


Test patch:
Repository: ESMCI/cime

Issue:
Output current machine and host with create_newcase and case.setup.
Add a helpful message about what CIME thinks the machine is to the output of both create_newcase and case.setup.  Something like "Machine is X, host is Y".   Output both for cases where someone specified a different machine then the host or the host allows multiple machines (like cori-knl, cori-haswell).



Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/case.py b/scripts/lib/CIME/case.py
--- a/scripts/lib/CIME/case.py
+++ b/scripts/lib/CIME/case.py
@@ -585,8 +585,14 @@ def configure(self, compset_name, grid_name, machine_name=None,
         #--------------------------------------------
         # set machine values in env_xxx files
         machobj = Machines(machine=machine_name)
+        probed_machine = machobj.probe_machine_name()
         machine_name = machobj.get_machine_name()
-        self.set_value("MACH",machine_name)
+        self.set_value("MACH", machine_name)
+        if probed_machine != machine_name and probed_machine is not None:
+            logger.warning("WARNING: User-selected machine '%s' does not match probed machine '%s'" % (machine_name, probed_machine))
+        else:
+            logger.info("Machine is %s" % machine_name)
+
         nodenames = machobj.get_node_names()
         nodenames =  [x for x in nodenames if
                       '_system' not in x and '_variables' not in x and 'mpirun' not in x and\


Test patch:
Repository: ESMCI/cime

Issue:
RUN_WITH_SUBMIT set to TRUE, but never set to FALSE
This is a minor issue that I just noticed. Caveat: I haven't done any testing; this just looks wrong to me. (This also appears to have been an issue in cime4.)

The RUN_WITH_SUBMIT xml variable is set to TRUE the first time you run case.submit. But it is never set to FALSE. I _think_ this means that the scripts will let you do the following:

```
case.submit
(wait for run to finish)
bsub < case.run
```

whereas they should yell at you for the attempt to resubmit without use of the case.submit script.

I think the fix for this would be, after this block in case_run:

```
    run_with_submit = case.get_value("RUN_WITH_SUBMIT")
    expect (run_with_submit,
            "You are not calling the run script via the submit script. "
            "As a result, short-term archiving will not be called automatically."
            "Please submit your run using the submit script like so:"
            " ./case.submit")
```

insert:

```
case.set_value("RUN_WITH_SUBMIT", False)
```

so that it's reset for the next time around.



Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/case_run.py b/scripts/lib/CIME/case_run.py
--- a/scripts/lib/CIME/case_run.py
+++ b/scripts/lib/CIME/case_run.py
@@ -245,6 +245,10 @@ def case_run(case):
            "Please submit your run using the submit script like so:"
            " ./case.submit")
 
+    # Forces user to use case.submit if they re-submit
+    if case.get_value("TESTCASE") is None:
+        case.set_value("RUN_WITH_SUBMIT", False)
+
     prerun_script = case.get_value("PRERUN_SCRIPT")
     postrun_script = case.get_value("POSTRUN_SCRIPT")
 


Test patch:
diff --git a/scripts/lib/CIME/case_test.py b/scripts/lib/CIME/case_test.py
--- a/scripts/lib/CIME/case_test.py
+++ b/scripts/lib/CIME/case_test.py
@@ -66,4 +66,6 @@ def case_test(case, testname=None):
 
     success = test.run()
 
+    case.set_value("RUN_WITH_SUBMIT", False)
+
     return success
Repository: ESMCI/cime

Issue:
Create a test for DLND
We need a test for DLND where DLND does something.


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/nmlgen.py b/scripts/lib/CIME/nmlgen.py
--- a/scripts/lib/CIME/nmlgen.py
+++ b/scripts/lib/CIME/nmlgen.py
@@ -326,7 +326,10 @@ def _sub_fields(self, varnames):
             if not line:
                 continue
             if "%glc" in line:
-                glc_nec_indices = range(self._case.get_value('GLC_NEC'))
+                if self._case.get_value('GLC_NEC') == 0:
+                    glc_nec_indices = [0]
+                else:
+                    glc_nec_indices = range(self._case.get_value('GLC_NEC'))
                 glc_nec_indices.append(glc_nec_indices[-1] + 1)
                 glc_nec_indices.pop(0)
                 for i in glc_nec_indices:


Test patch:
Repository: ESMCI/cime

Issue:
Invalid PIO numtasks not reset if stride is valid
The PIO numtasks are not reset if the strides are valid.

This is due to a copy-paste error in #817853704cd40d892fb711380922d991e127f4cb .
When copying code for adjust_pio_layout() in case_setup.py the last 3 lines were 
intended incorrectly


Discussion:
https://gist.github.com/jayeshkrishna/0e2231dba0a94df3109e80114c6368ed shows the relevant part of the change
The pio_numiotasks in the component namelists are set to -99 (instead of 1) for PET.f45_g37_rx1.A due to this bug.

Resolving patch:
diff --git a/scripts/lib/CIME/case_setup.py b/scripts/lib/CIME/case_setup.py
--- a/scripts/lib/CIME/case_setup.py
+++ b/scripts/lib/CIME/case_setup.py
@@ -237,9 +237,9 @@ def adjust_pio_layout(case, new_pio_stride):
         if pio_stride != new_stride:
             logger.info("Resetting  PIO_STRIDE_%s to %s"%(comp, new_stride))
             case.set_value("PIO_STRIDE_%s"%comp, new_stride)
-            if pio_numtasks != new_numtasks:
-                logger.info("Resetting  PIO_NUMTASKS_%s to %s"%(comp, new_numtasks))
-                case.set_value("PIO_NUMTASKS_%s"%comp, new_numtasks)
+        if pio_numtasks != new_numtasks:
+            logger.info("Resetting  PIO_NUMTASKS_%s to %s"%(comp, new_numtasks))
+            case.set_value("PIO_NUMTASKS_%s"%comp, new_numtasks)
 
 
 ###############################################################################


Test patch:
Repository: ESMCI/cime

Issue:
Short term archiver generates invalid characters in the log files
This issue is reproducible by running "create_test ERR.f45_g37_rx1.A" on anlworkstation.
In TestStatus.log there is output like below
```
st_archive starting
moving  /sandbox...
```
Between "moving" and "/sandbox" there is a space character, followed by \b (backspace, ASCII code 8)
If we use wait_for_tests script to generate Test.xml (based on TestStatus.log) and submit it to CDash, that xml file cannot be correctly parsed by CDash. If we open Test.xml with Firefox, it shows "XML Parsing Error: not well-formed" and refers to the location of the \b character.

In scripts/lib/CIME/case_st_archive.py, line 123 we have:
`logger.info("moving \b%s to \b%s" %(srcfile, destfile))`

Is there any reason to output \b to the log files? In line 171, it is using \n instead.
`logger.info("moving \n%s to \n%s " %(srcfile, destfile))`


Discussion:
That looks like a typo.  "b" and "n" are right next to each other.

Resolving patch:
diff --git a/scripts/lib/CIME/case_st_archive.py b/scripts/lib/CIME/case_st_archive.py
--- a/scripts/lib/CIME/case_st_archive.py
+++ b/scripts/lib/CIME/case_st_archive.py
@@ -43,11 +43,11 @@ def _get_ninst_info(case, compclass):
     ninst_strings = []
     if ninst is None:
         ninst = 1
-        for i in range(ninst):
-            if ninst > 1:
-                ninst_strings.append('_' + '%04d' % i)
-            else:
-                ninst_strings.append('')
+    for i in range(1,ninst+1):
+        if ninst > 1:
+            ninst_strings.append('_' + '%04d' % i)
+        else:
+            ninst_strings.append('')
 
     logger.debug("ninst and ninst_strings are: %s and %s for %s" %(ninst, ninst_strings, compclass))
     return ninst, ninst_strings
@@ -120,7 +120,7 @@ def _archive_log_files(case):
         srcfile = join(rundir, os.path.basename(logfile))
         destfile = join(archive_logdir, os.path.basename(logfile))
         shutil.move(srcfile, destfile)
-        logger.info("moving \b%s to \b%s" %(srcfile, destfile))
+        logger.info("moving \n%s to \n%s" %(srcfile, destfile))
 
 
 ###############################################################################
@@ -231,7 +231,7 @@ def _archive_restarts(case, archive, archive_entry,
         for i in range(ninst):
             restfiles = ""
             pattern = r"%s\.%s\d*.*" % (casename, compname)
-            if pattern != "dart":
+            if "dart" not in pattern:
                 pfile = re.compile(pattern)
                 files = [f for f in os.listdir(rundir) if pfile.search(f)]
                 if ninst_strings:


Test patch:
Repository: ESMCI/cime

Issue:
":" in test-id name causes error during build
 When I issue the command:
./create_test --xml-machine yellowstone --xml-compiler intel --xml-category aux_cam --test-id aux_cam_10:58:37

the job attempts to run, but there are misleading errors during compilation.  I would suggest that if the above name is not a valid name, then the error should be trapped and reported as such.  If it is a valid name, then a different fix will need to be made.

According to @gold2718 this behavior is also in the cime4, so it appears to be a bug that was migrated.



Discussion:
Is this a duplicate of https://github.com/CESM-Development/cime/issues/334 (but adding ':' to the list of disallowed characters), or is this a different issue (i.e., unrelated to the validity of the case name itself)?

I tested this again and this error still exists
@cacraigucar what was the recent command you tried to use? Trying to use the original command doesn't work due to unknown options it seems.

Resolving patch:
diff --git a/scripts/lib/CIME/utils.py b/scripts/lib/CIME/utils.py
--- a/scripts/lib/CIME/utils.py
+++ b/scripts/lib/CIME/utils.py
@@ -45,7 +45,7 @@ def check_name(fullname, additional_chars=None, fullpath=False):
     True
     """
 
-    chars = '<>/{}[\]~`@' # pylint: disable=anomalous-backslash-in-string
+    chars = '<>/{}[\]~`@:' # pylint: disable=anomalous-backslash-in-string
     if additional_chars is not None:
         chars += additional_chars
     if fullpath:


Test patch:
Repository: ESMCI/cime

Issue:
env_mach_pes.xml not being locked properly
I ran into this problem while doing testing on cheyenne.  I was able to make changes in env_mach_pes.xml and submit to the queue without any error messages.  

I was also able to change env_mach_pes.xml, then do a rebuild and submit without redoing case.setup.


Discussion:
I'm unable to reproduce this error locally (I don't have a cheyenne account), maybe it would help if I could get the git version and the exact commands and xml changes that were made.

And of course if this is only happening on cheyenne then someone else may have to look into it.

This is what I did using the current cime/master.

git version 1.8.5.6

./create_test SMS.f45_g37_rx1.A.cheyenne_intel
./xmlchange NTASKS=-2
./case.submit   # run submits to queue, but fails

./case.build --clean
./case.build
./case.submit   # run submits to queue, but fails

I wasn't able to get create_newcase to repeat this behavior.  So, it's something specific with create_test.
Thanks, that was very helpful. I think I am getting the same behavior now and will investigate.


NOTES:
Running create_newcase and changing the NTASKS gets the expected behavior:

./create_newcase --case test1 --res f45_g37_rx1 --mach anlworkstation --compiler gnu --compset A
cd test1
./case.setup
./case.build
./xmlchange NTASKS=-2
./case.submit
File /nfs2/sarich/cime4/scripts/test1/LockedFiles/env_mach_pes.xml has been modified
  found difference in NTASKS:{'component': 'ICE'} : case '-2' locked '-1'
  found difference in NTASKS:{'component': 'ESP'} : case '-2' locked '-1'
  found difference in NTASKS:{'component': 'CPL'} : case '-2' locked '-1'
  found difference in NTASKS:{'component': 'OCN'} : case '-2' locked '-1'
  found difference in NTASKS:{'component': 'LND'} : case '-2' locked '-1'
  found difference in NTASKS:{'component': 'ATM'} : case '-2' locked '-1'
  found difference in NTASKS:{'component': 'WAV'} : case '-2' locked '-1'
  found difference in NTASKS:{'component': 'GLC'} : case '-2' locked '-1'
  found difference in NTASKS:{'component': 'ROF'} : case '-2' locked '-1'


But using create_test for the same test case fails to catch the change:
./create_test SMS.f45_g37_rx1.A
cd ~/acme/scratch/SMS.f45_g37_rx1.A.anlworkstation_gnu.20170419_155053_r7bnhy
./xmlchange NTASKS=-2
./case.submit
...tries to run...



Resolving patch:
diff --git a/scripts/lib/CIME/check_lockedfiles.py b/scripts/lib/CIME/check_lockedfiles.py
--- a/scripts/lib/CIME/check_lockedfiles.py
+++ b/scripts/lib/CIME/check_lockedfiles.py
@@ -88,7 +88,7 @@ def check_lockedfiles(caseroot=None):
     for lfile in lockedfiles:
         fpart = os.path.basename(lfile)
         # ignore files used for tests such as env_mach_pes.ERP1.xml by looking for extra dots in the name
-        if lfile.count('.') > 1:
+        if fpart.count('.') > 1:
             continue
         cfile = os.path.join(caseroot, fpart)
         if os.path.isfile(cfile):


Test patch:
Repository: ESMCI/cime

Issue:
create_newcase needs --input-dir cla
the --input-dir argument recently added to create_test should also be added to create_newcase


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/case.py b/scripts/lib/CIME/case.py
--- a/scripts/lib/CIME/case.py
+++ b/scripts/lib/CIME/case.py
@@ -552,7 +552,8 @@ def configure(self, compset_name, grid_name, machine_name=None,
                   project=None, pecount=None, compiler=None, mpilib=None,
                   user_compset=False, pesfile=None,
                   user_grid=False, gridfile=None, ninst=1, test=False,
-                  walltime=None, queue=None, output_root=None, run_unsupported=False, answer=None):
+                  walltime=None, queue=None, output_root=None, run_unsupported=False, answer=None,
+                  input_dir=None):
 
         #--------------------------------------------
         # compset, pesfile, and compset components
@@ -821,6 +822,9 @@ def configure(self, compset_name, grid_name, machine_name=None,
         # Set TOTAL_CORES
         self.set_value("TOTAL_CORES", self.total_tasks * self.cores_per_task )
 
+        if input_dir is not None:
+            self.set_value("DIN_LOC_ROOT", os.path.abspath(input_dir))
+
     def get_compset_var_settings(self):
         compset_obj = Compsets(infile=self.get_value("COMPSETS_SPEC_FILE"))
         matches = compset_obj.get_compset_var_settings(self._compsetname, self._gridname)


Test patch:
diff --git a/scripts/lib/CIME/test_scheduler.py b/scripts/lib/CIME/test_scheduler.py
--- a/scripts/lib/CIME/test_scheduler.py
+++ b/scripts/lib/CIME/test_scheduler.py
@@ -365,6 +365,8 @@ def _create_newcase_phase(self, test):
             create_newcase_cmd += " --project %s " % self._project
         if self._output_root is not None:
             create_newcase_cmd += " --output-root %s " % self._output_root
+        if self._input_dir is not None:
+            create_newcase_cmd += " --input-dir %s " % self._input_dir
 
         if test_mods is not None:
             files = Files()
@@ -537,9 +539,6 @@ def _xml_phase(self, test):
             case.set_value("TEST", True)
             case.set_value("SAVE_TIMING", self._save_timing)
 
-            if self._input_dir is not None:
-                case.set_value("DIN_LOC_ROOT", self._input_dir)
-
         return True, ""
 
     ###########################################################################