Repository: ESMCI/cime

Issue:
You can't work from a directory that has tests in the name, it confuses CIME.
See https://github.com/E3SM-Project/E3SM/pull/6207#issuecomment-2038071022




Discussion:
I have run into this a few times when I make a workdirectory with either `tests` as part of a subdirectory name. I get an error something like:
```
$ ./create_test MVKO_PS.T62_oQU240.GMPAS-NYF -g /lcrc/group/e3sm/ac.xylar/e3sm_baselines/test_20240404 --pesfile ../../cime_config/testmods_dirs/config_pes_tests.xml --wait

Testnames: ['MVKO_PS.T62_oQU240.GMPAS-NYF.chrysalis_intel']
Using project from config_machines.xml: e3sm
create_test will do up to 1 tasks simultaneously
create_test will use up to 160 cores simultaneously
Creating test directory /lcrc/group/e3sm/ac.xasay-davis/scratch/chrys/MVKO_PS.T62_oQU240.GMPAS-NYF.chrysalis_intel.G.20240404_102105_tts9dt
RUNNING TESTS:
  MVKO_PS.T62_oQU240.GMPAS-NYF.chrysalis_intel
Starting CREATE_NEWCASE for test MVKO_PS.T62_oQU240.GMPAS-NYF.chrysalis_intel with 1 procs
Finished CREATE_NEWCASE for test MVKO_PS.T62_oQU240.GMPAS-NYF.chrysalis_intel in 0.175126 seconds (FAIL). [COMPLETED 1 of 1]
    Case dir: /lcrc/group/e3sm/ac.xasay-davis/scratch/chrys/MVKO_PS.T62_oQU240.GMPAS-NYF.chrysalis_intel.G.20240404_102105_tts9dt
    Errors were:
        ERROR: Makes no sense to have empty read-only file: /gpfs/fs1/home/ac.xylar/e3sm_work/E3SM/mkstratos/tests/add-mpaso-mvk-test/driver-nuopc/cime_config/config_component.xml
```

We have also seen this a few other times in the past:
https://github.com/E3SM-Project/E3SM/pull/5896#issuecomment-1703388921
https://github.com/E3SM-Project/E3SM/pull/5547#issuecomment-1483355470
@matthewhoffman just hit this issue:
```
$ ./create_test SMS_D_Ld1.T62_oEC60to30v3wLI_ais20.MPAS_LISIO_TEST.chrysalis_intel.mpaso-ocn_glcshelf
Testnames: ['SMS_D_Ld1.T62_oEC60to30v3wLI_ais20.MPAS_LISIO_TEST.chrysalis_intel.mpaso-ocn_glcshelf']
Using project from config_machines.xml: e3sm
create_test will do up to 1 tasks simultaneously
create_test will use up to 160 cores simultaneously
Creating test directory /lcrc/group/e3sm/ac.mhoffman/scratch/chrys/SMS_D_Ld1.T62_oEC60to30v3wLI_ais20.MPAS_LISIO_TEST.chrysalis_intel.mpaso-ocn_glcshelf.20240502_154158_9kugii
RUNNING TESTS:
  SMS_D_Ld1.T62_oEC60to30v3wLI_ais20.MPAS_LISIO_TEST.chrysalis_intel.mpaso-ocn_glcshelf
Starting CREATE_NEWCASE for test SMS_D_Ld1.T62_oEC60to30v3wLI_ais20.MPAS_LISIO_TEST.chrysalis_intel.mpaso-ocn_glcshelf with 1 procs
Finished CREATE_NEWCASE for test SMS_D_Ld1.T62_oEC60to30v3wLI_ais20.MPAS_LISIO_TEST.chrysalis_intel.mpaso-ocn_glcshelf in 0.553558 seconds (FAIL). [COMPLETED 1 of 1]
    Case dir: /lcrc/group/e3sm/ac.mhoffman/scratch/chrys/SMS_D_Ld1.T62_oEC60to30v3wLI_ais20.MPAS_LISIO_TEST.chrysalis_intel.mpaso-ocn_glcshelf.20240502_154158_9kugii
    Errors were:
        ERROR: Makes no sense to have empty read-only file: /gpfs/fs1/home/ac.mhoffman/e3sm-gis/E3SM-shelf-tests/driver-nuopc/cime_config/config_component.xml
```
It's a really common choice to have "test" or "tests" in the name of a directory you are developing tests in.
@jasonb5, I know you're a very busy guy but it would be a huge relief to have this fixed if you happen to find the time.

Resolving patch:
diff --git a/CIME/config.py b/CIME/config.py
--- a/CIME/config.py
+++ b/CIME/config.py
@@ -1,3 +1,4 @@
+import re
 import sys
 import glob
 import logging
@@ -39,9 +40,11 @@ def load(cls, customize_path):
 
         customize_files = glob.glob(f"{customize_path}/**/*.py", recursive=True)
 
+        ignore_pattern = re.compile(f"{customize_path}/(?:tests|conftest|test_)")
+
         # filter out any tests
         customize_files = [
-            x for x in customize_files if "tests" not in x and "conftest" not in x
+            x for x in customize_files if ignore_pattern.search(x) is None
         ]
 
         customize_module_spec = importlib.machinery.ModuleSpec("cime_customize", None)


Test patch:
diff --git a/CIME/tests/test_unit_config.py b/CIME/tests/test_unit_config.py
--- a/CIME/tests/test_unit_config.py
+++ b/CIME/tests/test_unit_config.py
@@ -1,11 +1,66 @@
 import os
 import unittest
 import tempfile
+from unittest import mock
+from pathlib import Path
 
 from CIME.config import Config
 
 
+def spy(method):
+    m = mock.MagicMock()
+
+    def wrapper(self, *args, **kwargs):
+        m(*args, **kwargs)
+        return method(self, *args, **kwargs)
+
+    wrapper.mock = m
+    return wrapper
+
+
 class TestConfig(unittest.TestCase):
+    def test_ignore(self):
+        test_paths = (
+            ("valid.py", False),
+            ("module/valid.py", False),
+            ("valid_test.py", False),
+            ("test_something.py", True),
+            ("tests/test_something.py", True),
+            ("conftest.py", True),
+            ("tests/conftest.py", True),
+            ("tests/generic/test_something.py", True),
+            ("tests/generic/conftest.py", True),
+        )
+
+        with tempfile.TemporaryDirectory() as _tempdir:
+            for src_path_name in ("generic", "test", "tests"):
+                customize_path = Path(
+                    _tempdir, src_path_name, "cime_config", "customize"
+                )
+
+                for test_path_name, _ in test_paths:
+                    test_file = customize_path / test_path_name
+
+                    test_file.parent.mkdir(parents=True, exist_ok=True)
+
+                    test_file.touch()
+
+                with mock.patch(
+                    "CIME.config.Config._load_file", spy(Config._load_file)
+                ) as mock_load_file:
+                    _ = Config.load(f"{customize_path}")
+
+                    loaded_files = [
+                        f'{Path(x[0][0]).relative_to(f"{customize_path}")}'
+                        for x in mock_load_file.mock.call_args_list
+                    ]
+
+                    for test_path_name, ignored in test_paths:
+                        if ignored:
+                            assert test_path_name not in loaded_files
+                        else:
+                            assert test_path_name in loaded_files
+
     def test_class_external(self):
         with tempfile.TemporaryDirectory() as tempdir:
             complex_file = os.path.join(tempdir, "01_complex.py")