Repository: ESMCI/cime

Issue:
PE layouts not being set correctly for user_compset without an alias
I have observed the following with the head of cime, but it was also reported with cesm2_0_beta02: If you create a case with a user_compset that doesn't have an alias, the PE layout is set incorrectly. As an example, here is a user_compset without a defined alias:

```
./create_newcase --user-compset -compset 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM1%NOEVOLVE_SWAV -res f09_g16 -case test_pes_1026g -mach yellowstone --walltime 02:00 --pesfile ../../components/clm/cime_config/config_pes.xml
Could not find a compset match for either alias or longname in 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM1%NOEVOLVE_SWAV
Pes setting: grid          is a%0.9x1.25_l%0.9x1.25_oi%gx1v6_r%r05_m%gx1v6_g%gland5UM_w%null
Pes setting: compset       is 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM1%NOEVOLVE_SWAV
Pes setting: grid match    is any
Pes setting: machine match is any
Pes setting: compset_match is any
Pes setting: pesize match  is any
Pes other settings: {'comment': 'default setting - used when no other match is found'}
```

whereas here is almost the exact same thing, but with a defined alias (just removed the %NOEVOLVE):

```
./create_newcase --user-compset -compset 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM1_SWAV -res f09_g16 -case test_pes_1026h -mach yellowstone --walltime 02:00 --pesfile ../../components/clm/cime_config/config_pes.xml
Compset longname is 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM1_SWAV
Compset specification file is /glade/scratch/sacks/cesm_code/cesm2_0_beta02_newCime/cime/../components/clm/cime_config/config_compsets.xml
Pes     specification file is /glade/scratch/sacks/cesm_code/cesm2_0_beta02_newCime/cime/../components/clm/cime_config/config_pes.xml
Pes setting: grid          is a%0.9x1.25_l%0.9x1.25_oi%gx1v6_r%r05_m%gx1v6_g%gland5UM_w%null
Pes setting: compset       is 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM1_SWAV
Pes setting: grid match    is l%0.9x1.25
Pes setting: machine match is yellowstone
Pes setting: compset_match is any
Pes setting: pesize match  is any
Pes other settings: {'comment': 'none'}
```

I'm pretty sure I remember this exact issue getting fixed before, but apparently it has gotten un-fixed.



Discussion:
This is still an issue in the latest cime. I tested this using a slight variation on the above:

This one doesn't have a defined alias:

```
[yslogin1:/glade/scratch/sacks/cesm_code/cesm2_0_alpha06i_newCime/cime/scripts]$ ./create_newcase --user-compset -compset 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM2%NOEVOLVE_SWAV -res f09_g16 -case test_pes_0421a --pesfile ../../components/clm/cime_config/config_pes.xml --run-unsupported
Could not find a compset match for either alias or longname in 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM2%NOEVOLVE_SWAV
Machine is yellowstone
Pes setting: grid          is a%0.9x1.25_l%0.9x1.25_oi%null_r%r05_g%gland4_w%null_m%gx1v6
Pes setting: compset       is 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM2%NOEVOLVE_SWAV
Pes setting: tasks       is {'NTASKS_ATM': -1, 'NTASKS_ICE': -1, 'NTASKS_CPL': -1, 'NTASKS_LND': -1, 'NTASKS_WAV': -1, 'NTASKS_ROF': -1, 'NTASKS_OCN': -1, 'NTASKS_GLC': -1}
Pes other settings: {'comment': 'default setting - used when no other match is found'}
 Compset is: 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM2%NOEVOLVE_SWAV
 Grid is: a%0.9x1.25_l%0.9x1.25_oi%null_r%r05_g%gland4_w%null_m%gx1v6
 Components in compset are: ['datm', 'clm', 'sice', 'socn', 'rtm', 'cism', 'swav', 'sesp', 'cpl', 'dart']
Using project from .cesm_proj: P93300601
cesm model version found: cesm2_0_alpha06i
 Creating Case directory /glade2/scratch2/sacks/cesm_code/cesm2_0_alpha06i_newCime/cime/scripts/test_pes_0421a
```

This one does:

```
[yslogin1:/glade/scratch/sacks/cesm_code/cesm2_0_alpha06i_newCime/cime/scripts]$ ./create_newcase --user-compset -compset 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM2_SWAV -res f09_g16 -case test_pes_0421b --pesfile ../../components/clm/cime_config/config_pes.xml --run-unsupported
Compset longname is 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM2_SWAV
Compset specification file is /glade2/scratch2/sacks/cesm_code/cesm2_0_alpha06i_newCime/cime/../components/clm/cime_config/config_compsets.xml
Pes     specification file is /glade2/scratch2/sacks/cesm_code/cesm2_0_alpha06i_newCime/cime/../components/clm/cime_config/config_pes.xml
Machine is yellowstone
Pes setting: grid match    is l%0.9x1.25
Pes setting: machine match is yellowstone
Pes setting: grid          is a%0.9x1.25_l%0.9x1.25_oi%null_r%r05_g%gland4_w%null_m%gx1v6
Pes setting: compset       is 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM2_SWAV
Pes setting: tasks       is {'NTASKS_ATM': 30, 'NTASKS_ICE': 600, 'NTASKS_CPL': 600, 'NTASKS_LND': 600, 'NTASKS_WAV': 600, 'NTASKS_ROF': 600, 'NTASKS_OCN': 600, 'NTASKS_GLC': 600}
Pes other settings: {'comment': 'none'}
 Compset is: 1850_DATM%CRU_CLM50%BGC_SICE_SOCN_RTM_CISM2_SWAV
 Grid is: a%0.9x1.25_l%0.9x1.25_oi%null_r%r05_g%gland4_w%null_m%gx1v6
 Components in compset are: ['datm', 'clm', 'sice', 'socn', 'rtm', 'cism', 'swav', 'sesp', 'cpl', 'dart']
Using project from .cesm_proj: P93300601
cesm model version found: cesm2_0_alpha06i
 Creating Case directory /glade2/scratch2/sacks/cesm_code/cesm2_0_alpha06i_newCime/cime/scripts/test_pes_0421b
```

I see:

```diff
[yslogin1:/glade/scratch/sacks/cesm_code/cesm2_0_alpha06i_newCime/cime/scripts]$ diff test_pes_0421a/env_mach_pes.xml test_pes_0421b
35c35
<     <entry id="TOTAL_CORES" value="15">
---
>     <entry id="TOTAL_CORES" value="630">
56,63c56,63
<         <value component="ATM">-1</value>
<         <value component="CPL">-1</value>
<         <value component="OCN">-1</value>
<         <value component="WAV">-1</value>
<         <value component="GLC">-1</value>
<         <value component="ICE">-1</value>
<         <value component="ROF">-1</value>
<         <value component="LND">-1</value>
---
>         <value component="ATM">30</value>
>         <value component="CPL">600</value>
>         <value component="OCN">600</value>
>         <value component="WAV">600</value>
>         <value component="GLC">600</value>
>         <value component="ICE">600</value>
>         <value component="ROF">600</value>
>         <value component="LND">600</value>
86c86
<         <value component="ATM">0</value>
---
>         <value component="ATM">600</value>
```

Resolving patch:
diff --git a/scripts/lib/CIME/XML/component.py b/scripts/lib/CIME/XML/component.py
--- a/scripts/lib/CIME/XML/component.py
+++ b/scripts/lib/CIME/XML/component.py
@@ -92,6 +92,17 @@ def _get_value_match(self, node, attributes=None, exact_match=False):
 
         return match_value
 
+    def get_description(self, compsetname):
+        rootnode = self.get_node("description")
+        desc_nodes = self.get_nodes("desc", root=rootnode)
+        desc = ""
+        for node in desc_nodes:
+            compsetmatch = node.get("compset")
+            if re.search(compsetmatch, compsetname):
+                desc += node.text
+
+        return desc
+
     def print_values(self):
         """
         print values for help and description in target config_component.xml file
diff --git a/scripts/lib/CIME/case.py b/scripts/lib/CIME/case.py
--- a/scripts/lib/CIME/case.py
+++ b/scripts/lib/CIME/case.py
@@ -72,7 +72,7 @@ def __init__(self, case_root=None, read_only=True):
         self._env_files_that_need_rewrite = set()
         self._read_only_mode = True
         self._force_read_only = read_only
-        self._principle_component = None
+        self._primary_component = None
 
         self._env_entryid_files = []
         self._env_generic_files = []
@@ -96,6 +96,7 @@ def __init__(self, case_root=None, read_only=True):
         self._gridfile = None
         self._components = []
         self._component_classes = []
+        self._component_description = {}
         self._is_env_loaded = False
         # these are user_mods as defined in the compset
         # Command Line user_mods are handled seperately
@@ -431,7 +432,7 @@ def _set_compset_and_pesfile(self, compset_name, files, user_compset=False, pesf
                 compsets = Compsets(compsets_filename)
                 match, compset_alias, science_support = compsets.get_compset_match(name=compset_name)
                 if match is not None:
-                    if pesfile is None:
+                    if self._pesfile is None:
                         self._pesfile = files.get_value("PES_SPEC_FILE"     , {"component":component})
                         self.set_lookup_value("PES_SPEC_FILE"      ,
                                               files.get_value("PES_SPEC_FILE"     , {"component":component}, resolved=False))
@@ -442,21 +443,19 @@ def _set_compset_and_pesfile(self, compset_name, files, user_compset=False, pesf
                     user_mods_dir     = files.get_value("USER_MODS_DIR"     , {"component":component}, resolved=False)
                     self.set_lookup_value("COMPSETS_SPEC_FILE" ,
                                    files.get_value("COMPSETS_SPEC_FILE", {"component":component}, resolved=False))
-                    self._principle_component = component.upper()
+                    self._primary_component = component.upper()
                     self.set_lookup_value("TESTS_SPEC_FILE"    , tests_filename)
                     self.set_lookup_value("TESTS_MODS_DIR"     , tests_mods_dir)
                     self.set_lookup_value("USER_MODS_DIR"      , user_mods_dir)
                     logger.info("Compset longname is %s"%(match))
                     logger.info("Compset specification file is %s" %(compsets_filename))
-                    logger.info("Pes     specification file is %s" %(pesfile))
+                    logger.info("Pes     specification file is %s" %(self._pesfile))
                     return compset_alias, science_support
 
         if user_compset is True:
             #Do not error out for user_compset
             logger.warn("Could not find a compset match for either alias or longname in %s" %(compset_name))
             self._compsetname = compset_name
-            logger.info("Pes     specification file is %s" %(pesfile))
-            self.set_lookup_value("PES_SPEC_FILE", pesfile)
         else:
             expect(False,
                    "Could not find a compset match for either alias or longname in %s\n"%(compset_name)
@@ -464,6 +463,58 @@ def _set_compset_and_pesfile(self, compset_name, files, user_compset=False, pesf
 
         return None, science_support
 
+    def _find_primary_component(self):
+        """
+        try to glean the primary component based on compset name
+        """
+        progcomps = {}
+        spec = {}
+        primary_component = None
+
+        for comp in self._component_classes:
+
+            if comp == "CPL":
+                continue
+            spec[comp] = self.get_value("COMP_%s"%comp)
+            notprogcomps = ("D%s"%comp,"X%s"%comp,"S%s"%comp)
+            if spec[comp].upper() in notprogcomps:
+                progcomps[comp] = False
+            else:
+                progcomps[comp] = True
+        expect("ATM" in progcomps and "LND" in progcomps and "OCN" in progcomps and \
+               "ICE" in progcomps, " Not finding expected components in %s"%self._component_classes)
+        if progcomps["ATM"] and progcomps["LND"] and progcomps["OCN"] and \
+           progcomps["ICE"]:
+            primary_component = "allactive"
+        elif progcomps["LND"] and progcomps["OCN"] and progcomps["ICE"]:
+            # this is a "J" compset
+            primary_component = "allactive"
+        elif progcomps["ATM"]:
+            if "DOCN%SOM" in self._compsetname:
+                # This is an "E" compset
+                primary_component = "allactive"
+            else:
+                # This is an "F" or "Q" compset
+                primary_component = spec["ATM"]
+        elif progcomps["LND"]:
+            # This is an "I" compset
+            primary_component = spec["LND"]
+        elif progcomps["OCN"]:
+            # This is a "C" or "G" compset
+            primary_component = spec["OCN"]
+        elif progcomps["ICE"]:
+            # This is a "D" compset
+            primary_component = spec["ICE"]
+        elif "GLC" in progcomps and progcomps["GLC"]:
+            # This is a "TG" compset
+            primary_component = spec["GLC"]
+        else:
+            # This is "A", "X" or "S"
+            primary_component = "drv"
+
+        return primary_component
+
+
     def get_compset_components(self):
         #If are doing a create_clone then, self._compsetname is not set yet
         components = []
@@ -545,9 +596,19 @@ def _get_component_config_data(self, files):
             expect(comp_config_file is not None and os.path.isfile(comp_config_file),
                    "Config file %s for component %s not found."%(comp_config_file, comp_name))
             compobj = Component(comp_config_file)
+            self._component_description[comp_class] = compobj.get_description(self._compsetname)
+            expect(self._component_description[comp_class] is not None,"No description found in file %s for component %s"%(comp_config_file, comp_name))
+            logger.info("%s component is %s"%(comp_class, self._component_description[comp_class]))
             for env_file in self._env_entryid_files:
                 env_file.add_elements_by_group(compobj, attributes=attlist)
 
+        if self._primary_component is None:
+            self._primary_component = self._find_primary_component()
+            if self._pesfile is None:
+                self._pesfile = files.get_value("PES_SPEC_FILE"     , {"component":self._primary_component})
+            logger.info("Pes specification file is %s" %(self._pesfile))
+            self.set_lookup_value("PES_SPEC_FILE", self._pesfile)
+
         # final cleanup of lookups table
         for key,value in self.lookups.items():
             result = self.set_value(key,value)
@@ -674,9 +735,6 @@ def configure(self, compset_name, grid_name, machine_name=None,
         compset_alias, science_support = self._set_compset_and_pesfile(compset_name, files, user_compset=user_compset, pesfile=pesfile)
 
         self._components = self.get_compset_components()
-        #FIXME - if --user-compset is True then need to determine that
-        #all of the compset settings are valid
-
         #--------------------------------------------
         # grid
         #--------------------------------------------
@@ -972,7 +1030,7 @@ def create_caseroot(self, clone=False):
         for newdir in newdirs:
             os.makedirs(newdir)
 
-        user_mods = self.get_value("%s_USER_MODS"%(self._principle_component))
+        user_mods = self.get_value("%s_USER_MODS"%(self._primary_component))
 
         # Open a new README.case file in $self._caseroot
         append_status(" ".join(sys.argv), "README.case", caseroot=self._caseroot)
@@ -991,8 +1049,11 @@ def create_caseroot(self, clone=False):
             if component_class == "CPL":
                 continue
             comp_grid = "%s_GRID"%component_class
+            append_status("Component %s is %s"%(component_class, self._component_description[component_class]),
+                          "README.case", caseroot=self._caseroot)
             append_status("%s is %s"%(comp_grid,self.get_value(comp_grid)),
                           "README.case", caseroot=self._caseroot)
+
         if user_mods is not None:
             note = "This compset includes user_mods %s"%user_mods
             append_status(note, "README.case", caseroot=self._caseroot)
@@ -1007,7 +1068,7 @@ def apply_user_mods(self, user_mods_dir=None):
         or they can be in the compset definition, or both.
         """
 
-        component_user_mods = self.get_value("%s_USER_MODS"%(self._principle_component))
+        component_user_mods = self.get_value("%s_USER_MODS"%(self._primary_component))
 
         # This looping order will lead to the specified user_mods_dir taking
         # precedence over self._user_mods, if there are any conflicts.


Test patch:
diff --git a/scripts/tests/scripts_regression_tests.py b/scripts/tests/scripts_regression_tests.py
--- a/scripts/tests/scripts_regression_tests.py
+++ b/scripts/tests/scripts_regression_tests.py
@@ -260,6 +260,48 @@ def test_a_createnewcase(self):
         run_cmd_assert_result(self, "./case.setup", from_dir=testdir)
         run_cmd_assert_result(self, "./case.build", from_dir=testdir)
 
+        with Case(testdir, read_only=True) as case:
+            case.set_comp_classes(case.get_values("COMP_CLASSES"))
+            primary = case._find_primary_component()
+            self.assertEqual(primary, "drv", msg="primary component test expected drv but got %s"%primary)
+            # now we are going to corrupt the case so that we can do more primary_component testing
+            case.set_valid_values("COMP_GLC","%s,fred"%case.get_value("COMP_GLC"))
+            case.set_value("COMP_GLC","fred")
+            primary = case._find_primary_component()
+            self.assertEqual(primary, "fred", msg="primary component test expected fred but got %s"%primary)
+            case.set_valid_values("COMP_ICE","%s,wilma"%case.get_value("COMP_ICE"))
+            case.set_value("COMP_ICE","wilma")
+            primary = case._find_primary_component()
+            self.assertEqual(primary, "wilma", msg="primary component test expected wilma but got %s"%primary)
+
+            case.set_valid_values("COMP_OCN","%s,bambam,docn"%case.get_value("COMP_OCN"))
+            case.set_value("COMP_OCN","bambam")
+            primary = case._find_primary_component()
+            self.assertEqual(primary, "bambam", msg="primary component test expected bambam but got %s"%primary)
+
+            case.set_valid_values("COMP_LND","%s,barney"%case.get_value("COMP_LND"))
+            case.set_value("COMP_LND","barney")
+            primary = case._find_primary_component()
+            # This is a "J" compset
+            self.assertEqual(primary, "allactive", msg="primary component test expected allactive but got %s"%primary)
+            case.set_value("COMP_OCN","docn")
+            case.set_valid_values("COMP_LND","%s,barney"%case.get_value("COMP_LND"))
+            case.set_value("COMP_LND","barney")
+            primary = case._find_primary_component()
+            self.assertEqual(primary, "barney", msg="primary component test expected barney but got %s"%primary)
+            case.set_valid_values("COMP_ATM","%s,wilma"%case.get_value("COMP_ATM"))
+            case.set_value("COMP_ATM","wilma")
+            primary = case._find_primary_component()
+            self.assertEqual(primary, "wilma", msg="primary component test expected wilma but got %s"%primary)
+            # this is a "E" compset
+            case._compsetname.replace("XOCN","DOCN%SOM")
+            primary = case._find_primary_component()
+            self.assertEqual(primary, "allactive", msg="primary component test expected allactive but got %s"%primary)
+            # finally a "B" compset
+            case.set_value("COMP_OCN","bambam")
+            primary = case._find_primary_component()
+            self.assertEqual(primary, "allactive", msg="primary component test expected allactive but got %s"%primary)
+
         cls._do_teardown.append(testdir)
 
     def test_b_user_mods(self):