Repository: popsim-consortium/stdpopsim (language: python)

Issue:
Problem with slim 3.6?
During the pytest transition in #777, I hit some problems from SLiM. This is probably not an issue with stdpopsim, but either pyslim or msprime. @petrelharp or @grahamgower, any chance one of you could look into this?

I worked around by pinning the CI tests to 3.5 - we should close this issue when we can run against 3.6.


Discussion / hints:
Looks like stdpopsim's slim engine code is relying on stdout/stderr behaviour of SLiM, which has changed in the 3.6 release.

Fix patch:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index a1bd62cfc..f50629a56 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -31,8 +31,8 @@ jobs:
             git clone --bare https://github.com/MesserLab/SLiM /tmp/SLiM_top/SLiM
             cd /tmp/SLiM_top/SLiM
             git describe --tags $(git rev-list --tags --max-count=1) > ${TAG_FILE}
-            # FIXME Pinning to SLiM 3.5 for now; looks like there's an issue with 3.6
-            echo v3.5 > ${TAG_FILE}
+            # Pinning to SLiM 3.6; comment out the next line to use latest tag.
+            echo v3.6 > ${TAG_FILE}
             echo Using SLiM version $(cat /tmp/SLiM_tag)
 
       - restore_cache:
diff --git a/stdpopsim/slim_engine.py b/stdpopsim/slim_engine.py
index f322e139c..7c6d68da4 100644
--- a/stdpopsim/slim_engine.py
+++ b/stdpopsim/slim_engine.py
@@ -1158,14 +1158,15 @@ def _run_slim(self, script_file, slim_path=None, seed=None, dry_run=False):
         By convention, messages from SLiM prefixed with "ERROR: " or
         "WARNING: " are treated as ERROR or WARN loglevels respectively.
         All other output on stdout is given the DEBUG loglevel.
-        ERROR messages, and any output from SLiM on stderr, will raise a
-        SLiMException here.
+        ERROR messages will raise a SLiMException here too, because
+        they are always generated by the `stop()` eidos function which
+        makes SLiM exit with a non-zero return code.
         """
         if slim_path is None:
             slim_path = self.slim_path()
 
-        # SLiM v3.5 needed with msprime 1.0
-        self._assert_min_version("3.5", slim_path)
+        # SLiM v3.6 sends `stop()` output to stderr, which we rely upon.
+        self._assert_min_version("3.6", slim_path)
 
         slim_cmd = [slim_path]
         if seed is not None:
@@ -1183,9 +1184,7 @@ def _run_slim(self, script_file, slim_path=None, seed=None, dry_run=False):
         ) as proc:
             for line in proc.stdout:
                 line = line.rstrip()
-                if line.startswith("ERROR: "):
-                    logger.error(line[len("ERROR: ") :])
-                elif line.startswith("WARNING: "):
+                if line.startswith("WARNING: "):
                     warnings.warn(
                         stdpopsim.UnspecifiedSLiMWarning(line[len("WARNING: ") :])
                     )
@@ -1193,9 +1192,13 @@ def _run_slim(self, script_file, slim_path=None, seed=None, dry_run=False):
                     # filter `dbg` function calls that generate output
                     line = line.replace("dbg(self.source); ", "")
                     logger.debug(line)
+
             stderr = proc.stderr.read()
+            for line in stderr.splitlines():
+                if line.startswith("ERROR: "):
+                    logger.error(line[len("ERROR: ") :])
 
-        if proc.returncode != 0 or stderr:
+        if proc.returncode != 0:
             raise SLiMException(
                 f"{slim_path} exited with code {proc.returncode}.\n{stderr}"
             )


Test patch:
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index f6b698b69..66ad24754 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -20,7 +20,7 @@ jobs:
         os: [ubuntu-18.04, windows-2019, macos-10.15]
         python: [3.7, 3.9]
     env:
-      SLIM_TAG: 3.5
+      SLIM_TAG: 3.6
       CONDA_ENV_NAME: stdpopsim
 
     steps: