Repository: ESMCI/cime

Issue:
New option for ERS tests?
Right now, if you run an ERS_Ld5 test then CIME does the following:

1. Run for 5 days, writing restart file after day 3
1. Go back to the day 3 restart file, run for 2 days (days 4 & 5)
1. Compare the output from step 1 and step 2

I'd like an option to run the test in the following manner:

1. Run for 5 days without writing a restart file
1. Run for 3 days (days 1 - 3) and write a restart file at the end of the run
1. Continue the run for 2 days (days 4 & 5)
1. Compare the output from step 1 and step 3

It is extra work (initializing the model three times instead of twice and running an additional three days), but POP has a "feature" where writing a restart history file changes the eventual history file<sup>1</sup> that is written... so if we have a test that computes 5-day averages for tavg output then the current version of the ERS test will `PASS` and the proposed option will `FAIL`. Having a test that fails in this situation would be handy for when we fix this feature.

<sup>1</sup> Two notes... the change in the history file is round-off level, so it isn't terrible that the test currently passes. Also, the change is due to the fact that the POP restart history file is a normalized version of the accumulation buffer so writing a restart file normalizes the buffer, writes it, and then denormalizes it (hence the pass -- in the current test step 1 denormalizes the buffer after _writing_ the restart and step 2 denormalizes it in an identical fashion after _reading_ the restart). The proposed fix to POP is to write the denormalized buffer into the restart history file.


Discussion:
My vote would be to make @mnlevy1981's 'option' the default usage. I've always been a bit suspicious that the current ERS test properly tests normal CESM restart usage.

Resolving patch:
diff --git a/scripts/lib/CIME/SystemTests/ers2.py b/scripts/lib/CIME/SystemTests/ers2.py
new file mode 100644
--- /dev/null
+++ b/scripts/lib/CIME/SystemTests/ers2.py
@@ -0,0 +1,60 @@
+"""
+CIME restart test 2  This class inherits from SystemTestsCommon
+"""
+from CIME.XML.standard_module_setup import *
+from CIME.SystemTests.system_tests_common import SystemTestsCommon
+
+logger = logging.getLogger(__name__)
+
+class ERS2(SystemTestsCommon):
+
+    def __init__(self, case):
+        """
+        initialize an object interface to the ERS2 system test
+        """
+        SystemTestsCommon.__init__(self, case)
+
+    def _ers2_first_phase(self):
+        stop_n      = self._case.get_value("STOP_N")
+        stop_option = self._case.get_value("STOP_OPTION")
+        rest_n      = self._case.get_value("REST_N")
+
+        # Don't need restarts for first run
+        self._case.set_value("REST_OPTION","never")
+
+        expect(stop_n > 0, "Bad STOP_N: %d" % stop_n)
+        expect(stop_n > 2, "ERROR: stop_n value %d too short"%stop_n)
+
+        logger.info("doing an %s %s initial test with restart file at %s %s"
+                    %(str(stop_n), stop_option, str(rest_n), stop_option))
+        self.run_indv()
+
+    def _ers2_second_phase(self):
+        stop_n      = self._case.get_value("STOP_N")
+        stop_option = self._case.get_value("STOP_OPTION")
+
+        rest_n = stop_n/2 + 1
+        stop_new = rest_n
+
+        self._case.set_value("REST_OPTION",stop_option)
+        self._case.set_value("STOP_N", stop_new)
+        self._case.flush()
+        logger.info("doing first part %s %s restart test"
+                    %(str(stop_new), stop_option))
+        self.run_indv(suffix="intermediate")
+
+        stop_new = stop_n - rest_n
+        self._case.set_value("STOP_N", stop_new)
+        self._case.set_value("CONTINUE_RUN", True)
+        self._case.set_value("REST_OPTION","never")
+
+        logger.info("doing second part %s %s restart test"
+                    %(str(stop_new), stop_option))
+        self.run_indv(suffix="rest")
+
+        # Compare restart file
+        self._component_compare_test("base", "rest")
+
+    def run_phase(self):
+        self._ers2_first_phase()
+        self._ers2_second_phase()


Test patch:
diff --git a/config/config_tests.xml b/config/config_tests.xml
--- a/config/config_tests.xml
+++ b/config/config_tests.xml
@@ -49,6 +49,13 @@ ERS    exact restart from startup (default 6 days + 5 days)
        if $IOP_ON is set then suffix is rest_iop
        compare component history files ".base" and ".rest" at day 11
 
+ERS2    exact restart from startup (default 6 days + 5 days)
+       do an 11 day initial test without making restarts     (suffix: base)
+       if $IOP_ON is set then suffix is base_iop
+       do an 11 day restart test stopping at day 6 with a restart, then resuming from restart at day 6 (suffix: rest)
+       if $IOP_ON is set then suffix is rest_iop
+       compare component history files ".base" and ".rest" at day 11
+
 ERP    pes counts hybrid (open-MP/MPI) restart bfb test from startup, default 6 days + 5 days (previousy PER)
        initial pes set up out of the box
        do an 11 day initial test - write a restart at day 6     (suffix base)
@@ -260,6 +267,19 @@ NODEFAIL          Tests restart upon detected node failure. Generates fake failu
     <CONTINUE_RUN>FALSE</CONTINUE_RUN>
   </test>
 
+  <test NAME="ERS2">
+    <DESC>exact restart from startup, default 6 days + 5 days</DESC>
+    <INFO_DBUG>1</INFO_DBUG>
+    <STOP_OPTION>ndays</STOP_OPTION>
+    <STOP_N>11</STOP_N>
+    <REST_N>$STOP_N / 2 + 1</REST_N>
+    <REST_OPTION>$STOP_OPTION</REST_OPTION>
+    <HIST_N>$STOP_N</HIST_N>
+    <HIST_OPTION>$STOP_OPTION</HIST_OPTION>
+    <DOUT_S>FALSE</DOUT_S>
+    <CONTINUE_RUN>FALSE</CONTINUE_RUN>
+  </test>
+
   <test NAME="ERIO">
     <DESC>exact restart from startup with different PIO methods, default 6 days + 5 days</DESC>
     <INFO_DBUG>1</INFO_DBUG>