internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ESMCI/cime

Issue:
nuopc run complete message location 
In commit b9d7b65fd case_run.py was changed to look for the run completed message in drv.log instead of in med.log
for nuopc runs, however the system_tests_common.py was not changed to correspond.   PR incoming.
@mvertens @uturuncoglu 


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/SystemTests/nodefail.py b/scripts/lib/CIME/SystemTests/nodefail.py
--- a/scripts/lib/CIME/SystemTests/nodefail.py
+++ b/scripts/lib/CIME/SystemTests/nodefail.py
@@ -24,7 +24,7 @@ def _restart_fake_phase(self):
         exeroot = self._case.get_value("EXEROOT")
         driver = self._case.get_value("COMP_INTERFACE")
         if driver == "nuopc":
-            logname = "med"
+            logname = "drv"
         else:
             logname = "cpl"
         fake_exe = \


Test patch:
diff --git a/scripts/lib/CIME/SystemTests/system_tests_common.py b/scripts/lib/CIME/SystemTests/system_tests_common.py
--- a/scripts/lib/CIME/SystemTests/system_tests_common.py
+++ b/scripts/lib/CIME/SystemTests/system_tests_common.py
@@ -33,7 +33,7 @@ def __init__(self, case, expected=None):
         self._init_environment(caseroot)
         self._init_locked_files(caseroot, expected)
         self._skip_pnl = False
-        self._cpllog = "med" if self._case.get_value("COMP_INTERFACE")=="nuopc" else "cpl"
+        self._cpllog = "drv" if self._case.get_value("COMP_INTERFACE")=="nuopc" else "cpl"
         self._ninja     = False
         self._dry_run   = False
         self._user_separate_builds = False
Repository: ESMCI/cime

Issue:
PR https://github.com/ESMCI/cime/pull/3884 broke a couple E3SM cases
    Errors were:
        ERROR: Expected one child, found None with name 'domains' and attribs '{'driver': 'mct'}' in file /home/jgfouca/E3SM/cime_config/config_grids.xml


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/XML/grids.py b/scripts/lib/CIME/XML/grids.py
--- a/scripts/lib/CIME/XML/grids.py
+++ b/scripts/lib/CIME/XML/grids.py
@@ -229,8 +229,10 @@ def _get_domains(self, component_grids, atmlevregex, lndlevregex, driver):
             domain_node = self.get_optional_child("domain", attributes={"name":grid_name_nonlev},
                                                   root=self.get_child("domains"))
             if not domain_node:
-                domain_node = self.get_optional_child("domain", attributes={"name":grid_name_nonlev},
-                                                      root=self.get_child("domains",{"driver":driver}))
+                domain_root = self.get_optional_child("domains",{"driver":driver})
+                if domain_root:
+                    domain_node = self.get_optional_child("domain", attributes={"name":grid_name_nonlev},
+                                                          root=domain_root)
             if domain_node:
                 comp_name = grid[0].upper()
 


Test patch:
Repository: ESMCI/cime

Issue:
Logic for maxinst in env_mach_pes fails with nuopc cases with python3
The maxinst expression in this block of code:

https://github.com/ESMCI/cime/blob/81d4f2033b5cbe7fb4c0b78fe71221b5e2182573/scripts/lib/CIME/XML/env_mach_pes.py#L105-L107

fails with python3:

```
Traceback (most recent call last):
  File "./create_newcase", line 244, in <module>
    _main_func(__doc__)
  File "./create_newcase", line 239, in _main_func
    extra_machines_dir=extra_machines_dir, case_group=case_group)
  File "/glade/work/sacks/ctsm_code/current_branch2/cime/scripts/Tools/../../scripts/lib/CIME/case/case.py", line 1672, in create
    extra_machines_dir=extra_machines_dir, case_group=case_group)
  File "/glade/work/sacks/ctsm_code/current_branch2/cime/scripts/Tools/../../scripts/lib/CIME/case/case.py", line 1120, in configure
    self.initialize_derived_attributes()
  File "/glade/work/sacks/ctsm_code/current_branch2/cime/scripts/Tools/../../scripts/lib/CIME/case/case.py", line 174, in initialize_derived_attributes
    self.total_tasks = env_mach_pes.get_total_tasks(comp_classes) + self.iotasks
  File "/glade/work/sacks/ctsm_code/current_branch2/cime/scripts/Tools/../../scripts/lib/CIME/XML/env_mach_pes.py", line 107, in get_total_tasks
    maxinst = max(maxinst, ninst)
TypeError: '>' not supported between instances of 'NoneType' and 'int'
```

Reproducer on cheyenne: From https://github.com/billsacks/ctsm/commit/bc4758979e36cbb6292962c5f65f9397797a4390

```
module load python
ncar_pylib
./create_newcase --case test_0407b --compset I2000Clm50BgcCropRtm --res f10_f10_mg37 --driver nuopc
```

It looks like python2 allows comparison between None and int, but python3 does not. The issue seems to be that ninst isn't defined per-component when running with nuopc, so ninst gets set to None here.


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/XML/env_mach_pes.py b/scripts/lib/CIME/XML/env_mach_pes.py
--- a/scripts/lib/CIME/XML/env_mach_pes.py
+++ b/scripts/lib/CIME/XML/env_mach_pes.py
@@ -102,7 +102,7 @@ def get_total_tasks(self, comp_classes):
             else:
                 nthrds = 1
 
-            if comp != "CPL" and (comp_interface!="nuopc" or not esmf_aware_threading):
+            if comp != "CPL" and comp_interface!="nuopc":
                 ninst = self.get_value("NINST", attribute={"compclass":comp})
                 maxinst = max(maxinst, ninst)
             tt = rootpe + nthrds*((ntasks - 1) * pstrid + 1)


Test patch:
Repository: ESMCI/cime

Issue:
Support for CESM containers - DIN_LOC_ROOT directory check
The CESM/CESM-Lab containers require a few minor modifications,including a minor change for convenience to how CIME handles the inputdata directory - currently, it fails if it doesn't exist, it seems we can simply change this to creating it when it doesn't exist without impacting existing systems, since their config entries already specify a shared input directory.  

(Changed this from two issues to one, since each will be handled by a different PR.)


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/case/case_setup.py b/scripts/lib/CIME/case/case_setup.py
--- a/scripts/lib/CIME/case/case_setup.py
+++ b/scripts/lib/CIME/case/case_setup.py
@@ -10,6 +10,7 @@
 from CIME.utils             import get_cime_root, run_and_log_case_status, get_model, get_batch_script_for_job, safe_copy
 from CIME.test_status       import *
 from CIME.locked_files      import unlock_file, lock_file
+import errno
 
 logger = logging.getLogger(__name__)
 
@@ -83,10 +84,18 @@ def _case_setup_impl(case, caseroot, clean=False, test_mode=False, reset=False,
     extra_machines_dir = case.get_value("EXTRA_MACHDIR")
     expect(mach is not None, "xml variable MACH is not set")
 
-    # Check that $DIN_LOC_ROOT exists - and abort if not a namelist compare tests
+    # Check that $DIN_LOC_ROOT exists or can be created:
     if not non_local:
         din_loc_root = case.get_value("DIN_LOC_ROOT")
         testcase     = case.get_value("TESTCASE")
+
+        if not os.path.isdir(din_loc_root):
+            try:
+                os.makedirs(din_loc_root)
+            except OSError as e:
+                if e.errno == errno.EACCES:
+                    logger.info("Invalid permissions to create {}".format(din_loc_root))
+            
         expect(not (not os.path.isdir(din_loc_root) and testcase != "SBN"),
                "inputdata root is not a directory or is not readable: {}".format(din_loc_root))
 


Test patch:
Repository: ESMCI/cime

Issue:
Active WAV component not recongized as primary component
In a new compset I created with an active WAV component and data ATM, the pe layout defaults to the datm pe specification file: (`src/drivers/mct/cime_config/config_pes.xml`)

I *think* this could be fixed by adding:
```
        elif progcomps["WAV"]:
            primary_component = spec["WAV"]
```
before the `else`  in  the following `if` statement:
https://github.com/ESMCI/cime/blob/44b241d263e3d2ace62f5340a1110f7174ca79e5/scripts/lib/CIME/case/case.py#L520-L573


Discussion:
The longname of the compset is:

`2000_DATM%IAF_SLND_SICE_SOCN_SROF_SGLC_WW3_SIAC_SESP`
@sbrus89 I think that you are correct.  Did you try it?
@jedwards4b, yes I did; it seems to work for me.
@sbrus89 please submit a PR with this change. We never had WAV as a primary component in the past. 
@sbrus89 - we had fixed this just a month or so ago? @jgfouca had put in that exact fix, so I'm not sure what happened
@sbrus89 -- I spoke too soon. We did exactly that but were adding the ability for ROF to be a primary component

@jedwards4b, will do. 

@jonbob, thanks for pointing that out; I can see that fix with blame. You're right that It'll be the exact same thing expect for WAV.

Resolving patch:
diff --git a/scripts/lib/CIME/case/case.py b/scripts/lib/CIME/case/case.py
--- a/scripts/lib/CIME/case/case.py
+++ b/scripts/lib/CIME/case/case.py
@@ -566,6 +566,9 @@ def _find_primary_component(self):
         elif progcomps["ROF"]:
             # This is a "R" compset
             primary_component = spec["ROF"]
+        elif progcomps["WAV"]:
+            # This is a "V" compset
+            primary_component = spec["WAV"]
         else:
             # This is "A", "X" or "S"
             primary_component = "drv"


Test patch:
Repository: ESMCI/cime

Issue:
cheyenne needs a module load python
Now that we require python 3.5+, we need to do a module load python on cheyenne.

The lack of this module load is responsible for a failure in `J_TestCreateNewcase.test_f_createnewcase_with_user_compset` if you run the whole `J_TestCreateNewcase` suite, and may cause other problems as well.

I'll get a fix in shortly.


Discussion:

Resolving patch:
diff --git a/scripts/Tools/standard_script_setup.py b/scripts/Tools/standard_script_setup.py
--- a/scripts/Tools/standard_script_setup.py
+++ b/scripts/Tools/standard_script_setup.py
@@ -14,6 +14,6 @@
 os.environ["CIMEROOT"] = _CIMEROOT
 
 import CIME.utils
-CIME.utils.check_minimum_python_version(2, 7)
+CIME.utils.check_minimum_python_version(3, 6)
 CIME.utils.stop_buffering_output()
 import logging, argparse


Test patch:
Repository: ESMCI/cime

Issue:
Invalid job name for PBS queue 
When running a test like ERS_Vnuopc.T62_g37.2000_DATM%NYF_SLND_CICE6_POP2_DROF%NYF_SGLC_WW3.cheyenne_gnu.pop-cice 

The following job name is created.

#PBS -N test.ERS_Vnuopc.T62_g37.2000_DATM%NYF_SLND_CICE6_POP2_DROF%NYF_SGLC_WW3.cheyenne_gnu.pop-cice.20210804_174443_j8gvpu

PBS doesn't like the % and throws an error of

ERROR: Command: 'qsub -q regular -l walltime=12:00:00 -A P93300606  .case.test' failed with error 'qsub: illegal -N value.

We should probably strip special characters from the job_id that's used in env_batch.xml.


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/XML/env_batch.py b/scripts/lib/CIME/XML/env_batch.py
--- a/scripts/lib/CIME/XML/env_batch.py
+++ b/scripts/lib/CIME/XML/env_batch.py
@@ -198,7 +198,13 @@ def make_batch_script(self, input_template, job, case, outfile=None):
             ext = job
         if ext.startswith('.'):
             ext = ext[1:]
-        overrides["job_id"] = ext + '.' + case.get_value("CASE")
+
+        # A job name or job array name can be at most 230 characters.  It must consist only of alphabetic, numeric, plus 
+        # sign ("+"), dash or minus or hyphen ("-"), underscore ("_"), and dot or period (".") characters
+        # most of these are checked in utils:check_name, but % is not one of them.
+
+        overrides["job_id"] = ext + '.' + case.get_value("CASE").replace('%','')
+
         overrides["batchdirectives"] = self.get_batch_directives(case, job, overrides=overrides)
         output_text = transform_vars(open(input_template,"r").read(), case=case, subgroup=job, overrides=overrides)
         output_name = get_batch_script_for_job(job) if outfile is None else outfile


Test patch:
Repository: ESMCI/cime

Issue:
ftp server error 
The ftp server at ftp://ftp.cgd.ucar.edu/cesm/inputdata is currently down, this has exposed an
issue handling errors in check_input_data.py fix in the works.  


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/Servers/ftp.py b/scripts/lib/CIME/Servers/ftp.py
--- a/scripts/lib/CIME/Servers/ftp.py
+++ b/scripts/lib/CIME/Servers/ftp.py
@@ -48,7 +48,13 @@ def ftp_login(cls, address, user='', passwd=''):
         except RuntimeError:
             logger.warning("ftp login timeout!")
             return None
-        return cls(address, user=user, passwd=passwd, server=ftp)
+        result = None
+        try:
+            result = cls(address, user=user, passwd=passwd, server=ftp)
+        except all_ftp_errors as e:
+            logger.warning("ftp error: {}".format(e))
+
+        return result
 
     def fileexists(self, rel_path):
         try:
diff --git a/scripts/lib/CIME/case/check_input_data.py b/scripts/lib/CIME/case/check_input_data.py
--- a/scripts/lib/CIME/case/check_input_data.py
+++ b/scripts/lib/CIME/case/check_input_data.py
@@ -164,16 +164,17 @@ def check_all_input_data(self, protocol=None, address=None, input_data_root=None
         if chksum:
             chksum_found = _download_checksum_file(self.get_value("RUNDIR"))
 
-        success = self.check_input_data(protocol=protocol, address=address, download=False,
+        clm_usrdat_name = self.get_value("CLM_USRDAT_NAME")
+        if download and clm_usrdat_name:
+            success = _downloadfromserver(self, input_data_root, data_list_dir,
+                                          attributes={"CLM_USRDAT_NAME":clm_usrdat_name})
+        if not success:
+            success = self.check_input_data(protocol=protocol, address=address, download=False,
                                         input_data_root=input_data_root, data_list_dir=data_list_dir, chksum=chksum and chksum_found)
         if download and not success:
             if not chksum:
                 chksum_found = _download_checksum_file(self.get_value("RUNDIR"))
             success = _downloadfromserver(self, input_data_root, data_list_dir)
-            clm_usrdat_name = self.get_value("CLM_USRDAT_NAME")
-            if not success and clm_usrdat_name:
-                success = _downloadfromserver(self, input_data_root, data_list_dir,
-                                              attributes={"CLM_USRDAT_NAME":clm_usrdat_name})
 
     expect(not download or (download and success), "Could not find all inputdata on any server")
     self.stage_refcase(input_data_root=input_data_root, data_list_dir=data_list_dir)


Test patch:
Repository: ESMCI/cime

Issue:
PR #4106 causes failures in auto-detecting some machine names
The changes in #4106 cause failures in auto-detecting some machine names, resulting in the following (at least for CESM):

```
ERROR: No machine $ENV{NERSC_HOST}:perlmutter found
```

The problem arises when the first call to `_probe_machine_name_one_guess` from `probe_machine_name` fails (because `socket.getfqdn()` doesn't provide the match, but the later call to `socket.gethostname()` will provide the match). The specific issue is that the `machine` variable was used for multiple purposes in the changes to `machines.py` in #4106, so when `_probe_machine_name_one_guess` was supposed to return `None`, it was instead returning the text value of `NODENAME_REGEX` from the last machine with an `$ENV{XXX}:YYY` pattern.

I have a fix on a branch and will open a PR shortly.


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/XML/machines.py b/scripts/lib/CIME/XML/machines.py
--- a/scripts/lib/CIME/XML/machines.py
+++ b/scripts/lib/CIME/XML/machines.py
@@ -175,13 +175,13 @@ def _probe_machine_name_one_guess(self, nametomatch):
                 logger.debug("machine regex string is " + regex_str)
                 # an environment variable can be used
                 if regex_str.startswith("$ENV"):
-                    machine = self.get_resolved_value(regex_str, allow_unresolved_envvars=True)
-                    if not machine.startswith("$ENV"):
+                    machine_value = self.get_resolved_value(regex_str, allow_unresolved_envvars=True)
+                    if not machine_value.startswith("$ENV"):
                         try:
-                            match,machine = machine.split(":")
+                            match, this_machine = machine_value.split(":")
                         except ValueError:
                             expect(False,"Bad formation of NODENAME_REGEX.  Expected envvar:value, found {}".format(regex_str))
-                        if match == machine:
+                        if match == this_machine:
                             machine = machtocheck
                             break
                 else:    


Test patch:
Repository: ESMCI/cime

Issue:
Overwriting baselines should not leave stale files
The directory should probably be blown away and re-created to avoid leaving stale files that can cause confusion. See:
https://github.com/E3SM-Project/E3SM/issues/3374



Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/utils.py b/scripts/lib/CIME/utils.py
--- a/scripts/lib/CIME/utils.py
+++ b/scripts/lib/CIME/utils.py
@@ -2569,3 +2569,17 @@ def get_urlroot(machobj=None):
 
     logger.info("No urlroot info available")
     return None
+
+
+def clear_folder(_dir):
+    if os.path.exists(_dir):
+        for the_file in os.listdir(_dir):
+            file_path = os.path.join(_dir, the_file)
+            try:
+                if os.path.isfile(file_path):
+                    os.unlink(file_path)
+                else:
+                    clear_folder(file_path)
+                    os.rmdir(file_path)
+            except Exception as e:
+                print(e)


Test patch:
diff --git a/scripts/lib/CIME/test_scheduler.py b/scripts/lib/CIME/test_scheduler.py
--- a/scripts/lib/CIME/test_scheduler.py
+++ b/scripts/lib/CIME/test_scheduler.py
@@ -27,6 +27,7 @@
     get_timestamp,
     get_python_libs_root,
     get_cime_default_driver,
+    clear_folder,
 )
 from CIME.test_status import *
 from CIME.XML.machines import Machines
@@ -339,7 +340,8 @@ def __init__(
                     test_baseline = os.path.join(full_baseline_dir, test_name)
                     if os.path.isdir(test_baseline):
                         existing_baselines.append(test_baseline)
-
+                        if allow_baseline_overwrite:
+                            clear_folder(test_baseline)
                 expect(
                     allow_baseline_overwrite or len(existing_baselines) == 0,
                     "Baseline directories already exists {}\n"
Repository: ESMCI/cime

Issue:
Consider removing test_release category from case.py
The test_release category was created for a one-time use, but can be removed as no one actively uses it for testing.

We are removing/moving all test_release tests in CAM.

in sum:
"test_release" can be removed from the following line in case.py (in _check_testlists):
`test_categories = ["prealpha", "prebeta", "test_release"]`


Discussion:

Resolving patch:
diff --git a/scripts/lib/CIME/case/case.py b/scripts/lib/CIME/case/case.py
--- a/scripts/lib/CIME/case/case.py
+++ b/scripts/lib/CIME/case/case.py
@@ -2028,7 +2028,7 @@ def _check_testlists(self, compset_alias, grid_name, files):
             testlist = tests.get_tests(
                 compset=compset_alias, grid=grid_name, supported_only=True
             )
-            test_categories = ["prealpha", "prebeta", "test_release"]
+            test_categories = ["prealpha", "prebeta"]
             for test in testlist:
                 if (
                     test["category"] in test_categories


Test patch:
Repository: ESMCI/cime

Issue:
CAM SE grids are not fully defined in case 
In particular test ERS.ne5_ne5_mg37.FADIAB.cheyenne_intel.cam-outfrq3s_bwic does not
correctly define ATM_GRID.   


Discussion:

Resolving patch:
diff --git a/CIME/XML/grids.py b/CIME/XML/grids.py
--- a/CIME/XML/grids.py
+++ b/CIME/XML/grids.py
@@ -312,9 +312,6 @@ def _get_domains_for_one_grid(
                     domains[comp_name + "_NX"] = 1
                     domains[comp_name + "_NY"] = 1
 
-            # set up dictionary of domain files for every component
-            _add_grid_info(domains, comp_name + "_GRID", grid_name)
-
             if driver == "mct":
                 # mct
                 file_nodes = self.get_children("file", root=domain_node)
@@ -360,6 +357,8 @@ def _get_domains_for_one_grid(
                         file_node = self.get_optional_child("file", root=domain_node)
                         if file_node is not None and self.text(file_node) != "unset":
                             domains["PTS_DOMAINFILE"] = self.resolved_text(file_node)
+        # set up dictionary of domain files for every component
+        _add_grid_info(domains, comp_name + "_GRID", grid_name)
 
     def _get_gridmaps(self, component_grids, driver, compset):
         """Set all mapping files for config_grids.xml v2 schema


Test patch:
Repository: ESMCI/cime

Issue:
Remove argparse.SUPPRESS for --driver option in create_newcase
The --driver option is suppressed in the help for create_newcase. It is available for create_test. For CESM it would be nice to have help for it so that users can see that it's an option, especially for our transition from MCT to NUOPC. I assume the only reason it was suppressed was it was considered an advanced feature that most people wouldn't use.

There's two other options that are suppressed --script-root and --test and for those I see why they are suppressed.

The change is just to remove the help=argparse.SUPPRESS and add a help string such as

"Override the top level driver type and use this one (changes xml variable COMP_INTERFACE) [this is an advanced option]"


Discussion:
@jgfouca you are the one that added this in, and included the suppress. What do you think about adding help for this?
@ekluzek what would be ideal is for only model 'cesm' to have this unsuppressed because we (e3sm) still do not support this option.

Resolving patch:
diff --git a/CIME/scripts/create_newcase.py b/CIME/scripts/create_newcase.py
--- a/CIME/scripts/create_newcase.py
+++ b/CIME/scripts/create_newcase.py
@@ -231,12 +231,25 @@ def parse_command_line(args, cimeroot, description):
         "--input-dir",
         help="Use a non-default location for input files. This will change the xml value of DIN_LOC_ROOT.",
     )
-    parser.add_argument(
-        "--driver",
-        default=get_cime_default_driver(),
-        choices=("mct", "nuopc", "moab"),
-        help=argparse.SUPPRESS,
-    )
+    if model == "cesm":
+        drv_choices = ("mct", "nuopc")
+        drv_help = (
+            "Override the top level driver type and use this one "
+            + "(changes xml variable COMP_INTERFACE) [this is an advanced option]"
+        )
+    elif model == "e3sm":
+        drv_choices = ("mct", "moab")
+        drv_help = argparse.SUPPRESS
+    else:
+        drv_choices = None
+
+    if drv_choices is not None:
+        parser.add_argument(
+            "--driver",
+            default=get_cime_default_driver(),
+            choices=drv_choices,
+            help=drv_help,
+        )
 
     parser.add_argument(
         "-n",


Test patch: