Repository: LUMC/pytest-workflow (language: python)

Issue:
Fix symlinks to directories fail when copying with --git-aware flag.
### Checklist
- [x] Pull request details were added to HISTORY.rst

Fixes #162


Discussion / hints:

Fix patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 11d7c966..88eb8b00 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -14,7 +14,7 @@ jobs:
     strategy:
       matrix:
         python-version:
-          - "3.6"
+          - "3.7"
     steps:
       - uses: actions/checkout@v2.3.4
       - name: Set up Python ${{ matrix.python-version }}
@@ -26,7 +26,6 @@ jobs:
       - name: Lint
         run: tox -e lint
   docs:
-    needs: lint
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v2.3.4
@@ -43,12 +42,11 @@ jobs:
     strategy:
       matrix:
         python-version:
-          - "3.6"
           - "3.7"
           - "3.8"
           - "3.9"
           - "3.10"
-    needs: lint
+          - "3.11"
     steps:
       - uses: actions/checkout@v2.3.4
       - name: Set up Python ${{ matrix.python-version }}
@@ -60,7 +58,6 @@ jobs:
       - name: Run tests
         run: tox -e py3
       - name: Upload coverage report
-        if: ${{ matrix.python-version == 3.6 }} # Only upload coverage once
         uses: codecov/codecov-action@v1
 
   test-functional:
diff --git a/HISTORY.rst b/HISTORY.rst
index dfd74e98..79d9d35c 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -7,6 +7,45 @@ Changelog
 .. This document is user facing. Please word the changes in such a way
 .. that users understand how the changes affect the new version.
 
+version 2.0.0
+---------------------------
+This major release greatly cleans up the output of pytest-workflow in case of
+an error as well as providing the stderr and stdout last bytes for debugging
+purposes. When the exit code test fails all other tests from the workflow
+are skipped.
+
++ Python 3.6 is no longer supported. It has been removed from github actions,
+  as such we can no longer guarantee that pytest-workflow works properly
+  with python 3.6.
++ Fix an issue where symlinks in git repositories could not be properly copied.
++ Added an optional encoding key for files, stdout and stderr so the file can
+  be opened with the proper encoding.
++ Make content tests more efficient by reading each file only once instead of
+  twice when there are both strings and regexes to check for.
++ When the ``--git-aware`` flag is used a submodule check is performed in order
+  to assert that all submodules are properly checked out. This prevents
+  unclear copying errors.
++ Tests are now skipped if the workflow does not exit with the desired exit
+  code, except for the exit code tests, to reduce visual clutter when reporting
+  failing tests.
++ Tests for checking file content are now skipped when the file does not exist
+  in order to reduce visual clutter when reporting failing tests.
++ Test and support for Python 3.11.
++ Add ``--stderr-bytes`` or ``--sb`` option to change the maximum 
+  number of bytes to display for the stderr and stdout on 
+  command failure. 
++ Add stderr and stdout to be displayed on command failure
++ Document using ``pytest.ini`` as a way of setting specific per repository
+  settings for pytest-workflow.
++ Add tests for nextflow.
++ Document the use of environment variables with pytest-workflow.
++ A minimum of pytest 7.0.0 is now a requirement for pytest-workflow.
+  This fixes the deprecation warnings that started on the release of pytest
+  7.0.0.
++ Throw a more descriptive error when a file copied with the --git-aware flag
+  is not present on the filesystem anymore.
++ Document pytest flags ``-k`` and ``-m`` are not supported.
+
 version 1.6.0
 ---------------------------
 + Add a ``--git-aware`` or ``--ga`` option to only copy copy files listed by
diff --git a/README.rst b/README.rst
index a70ead6c..c53eb866 100644
--- a/README.rst
+++ b/README.rst
@@ -34,18 +34,18 @@ pytest-workflow
 
 pytest-workflow is a workflow-system agnostic testing framework that aims 
 to make pipeline/workflow testing easy by using YAML files for the test 
-configuration. Whether you write your pipelines in WDL, snakemake, bash or
-any other workflow framework, pytest-workflow makes testing easy.
+configuration. Whether you write your pipelines in WDL, snakemake, nextflow, 
+bash or any other workflow framework, pytest-workflow makes testing easy.
 pytest-workflow is build on top of the pytest test framework.
 
-For our complete documentation checkout our
+For our complete documentation and examples checkout our
 `readthedocs page <https://pytest-workflow.readthedocs.io/>`_.
 
 
 Installation
 ============
-Pytest-workflow requires Python 3.6 or higher. It is tested on Python 3.6, 3.7,
-3.8, 3.9 and 3.10. Python 2 is not supported.
+Pytest-workflow requires Python 3.7 or higher. It is tested on Python 3.7,
+3.8, 3.9, 3.10 and 3.11.
 
 - Make sure your virtual environment is activated.
 - Install using pip ``pip install pytest-workflow``
@@ -103,6 +103,7 @@ predefined tests as well as custom tests are possible.
         must_not_contain:              # A list of strings that should NOT be in the file (optional)
           - "Cock a doodle doo"
         md5sum: e583af1f8b00b53cda87ae9ead880224   # Md5sum of the file (optional)
+        encoding: UTF-8                # Encoding for the text file (optional). Defaults to system locale.
 
   - name: simple echo                  # A second workflow. Notice the starting `-` which means
     command: "echo moo"                # that workflow items are in a list. You can add as much workflows as you want
@@ -114,6 +115,7 @@ predefined tests as well as custom tests are possible.
         - "moo"
       must_not_contain:                # List of strings that should NOT be in stout (optional)
         - "Cock a doodle doo"
+      encoding: ASCII                  # Encoding for stdout (optional). Defaults to system locale.
 
   - name: mission impossible           # Also failing workflows can be tested
     tags:                              # A list of tags that can be used to select which test
@@ -130,6 +132,7 @@ predefined tests as well as custom tests are possible.
         - "BSOD error, please contact the IT crowd"
       must_not_contain:                # A list of strings which should NOT be in stderr (optional)
         - "Mission accomplished!"
+      encoding: UTF-16                 # Encoding for stderr (optional). Defaults to system locale.
 
   - name: regex tests
     command: echo Hello, world
@@ -143,7 +146,7 @@ predefined tests as well as custom tests are possible.
         - '^Hello .*'                  # Complex regexes will break yaml if double quotes are used
 
 For more information on how Python parses regular expressions, see the `Python
-documentation <https://docs.python.org/3.6/library/re.html>`_.
+documentation <https://docs.python.org/3/library/re.html>`_.
 
 Documentation for more advanced use cases including the custom tests can be
 found on our `readthedocs page <https://pytest-workflow.readthedocs.io/>`_.
diff --git a/docs/conf.py b/docs/conf.py
index 873b5186..41038374 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -64,7 +64,7 @@
 #
 # This is also used if you do content translation via gettext catalogs.
 # Usually you set "language" from the command line for these cases.
-language = None
+language = "en"
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
diff --git a/docs/examples.rst b/docs/examples.rst
index 166e7e22..55947c55 100644
--- a/docs/examples.rst
+++ b/docs/examples.rst
@@ -79,14 +79,12 @@ information on the localization of output files as well as options to modify
 the running of miniwdl from the environment.
 
 Miniwdl will localize all the output files to an ``output_links`` directory
-inside the test output directory. If you have a workflow with the output:
+inside the test output directory. If you have a workflow with the output::
 
-.. code-block::
-
-        output {
-            File moo_file = moo_task.out
-            Array[File] stats = moo_task.stats_files
-        }
+    output {
+        File moo_file = moo_task.out
+        Array[File] stats = moo_task.stats_files
+    }
 
 Inside the ``out`` directory the directories ``moo_file`` and
 ``stats`` will be created. Inside these directories will be the produced files.
@@ -109,3 +107,96 @@ The following yaml file tests a WDL pipeline run with miniwdl.
 
 Please note that the trailing slash in ``-d test-output/`` is important. It
 will ensure the files end up in the ``test-output`` directory.
+
+Nextflow example
+-----------------
+
+With nextflow each process is run in a unique directory where the output files will
+also be stored. Nextflow can output a copy of the output files to a separate workflow-outputs 
+directory. This can be achieved by defining a ``publishDir`` in the process. Through ``params.outdir``
+it is possible to define the output directory when running the code.
+
+An example code defining a ``publishDir`` is listed below. ::
+
+    process Hello {
+        publishDir = [
+            path: { "${params.outdir}/hello"}
+        ]
+
+        output:
+        path "HelloWorld.txt"
+        script:
+        """
+        echo "Hello World!" > HelloWorld.txt
+        """
+    }
+
+    workflow {
+        Hello
+    }
+
+To run the code listed above the following command can be used in which ``examplecode.nf`` is the code listed above:
+
+.. code-block:: bash
+
+    nextflow run examplecode.nf --outdir test-output
+
+``publishDir`` will make it so that all the output files of the process are copied to the given directory. 
+``--outdir`` is used to define the path the output files will go to. In this case ``HelloWorld.txt`` will 
+be copied to the  directory called ``test-output/hello``.
+
+An example yaml file that could be used to test the nextflow pipeline from ``examplecode.nf`` is listed
+below.
+
+.. code-block:: yaml
+
+    - name: My pipeline
+      command: nextflow run examplecode.nf --outdir test-output
+      files:
+        - path: "test-output/hello/HelloWorld.txt"
+
+Bash example
+------------
+
+The following is an example of a Bash file that can run directly as a script, or sourced to test each function separately:
+
+.. code-block:: bash
+
+    #!/usr/bin/env bash
+
+    function say_hello() {
+        local name="$1"
+        echo "Hello, ${name}!"
+    }
+
+    function main() {
+        say_hello world
+    }
+
+    # Only execute main when this file is run as a script
+    if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
+        main
+    fi
+
+Save the bash file as ``script.sh``, and test it with the following pytest-workflow configuration:
+
+
+.. code-block:: yaml
+
+    - name: test bash script
+      command: bash script.sh
+      stdout:
+        contains:
+          - "Hello, world!"
+
+    - name: test bash function
+      command: >
+        bash -c "
+        source script.sh;
+        say_hello pytest-workflow
+        "
+      stdout:
+        contains:
+          - "Hello, pytest-workflow!"
+        must_not_contain:
+          - "Hello, world!"
diff --git a/docs/installation.rst b/docs/installation.rst
index 1a38d3bf..d07b87db 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -2,8 +2,7 @@
 Installation
 ============
 
-Pytest-workflow is tested on python 3.6, 3.7, 3.8, 3.9 and 3.10. Python 2 is not
-supported.
+Pytest-workflow is tested on python 3.7, 3.8, 3.9, 3.10 and 3.11.
 
 In a virtual environment
 ------------------------
diff --git a/docs/known_issues.rst b/docs/known_issues.rst
index 42808670..c6ec3857 100644
--- a/docs/known_issues.rst
+++ b/docs/known_issues.rst
@@ -21,4 +21,4 @@ Known issues
   ``contains_regex`` and ``must_not_contain_regex``, since this collides with
   Python's usage of the same character to escape special characters in strings.
   Please see the `Python documentation on regular expressions
-  <https://docs.python.org/3.6/library/re.html>`_ for details.
+  <https://docs.python.org/3/library/re.html>`_ for details.
diff --git a/requirements.txt b/requirements.txt
index be8fac62..2c5d3bff 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,3 +1,3 @@
 pyyaml
-pytest>=5.4.0
+pytest>=7.0.0
 jsonschema
\ No newline at end of file
diff --git a/setup.py b/setup.py
index 8771bba5..50d383c0 100644
--- a/setup.py
+++ b/setup.py
@@ -20,7 +20,7 @@
 
 setup(
     name="pytest-workflow",
-    version="1.6.0",
+    version="2.0.0",
     description="A pytest plugin for configuring workflow/pipeline tests "
                 "using YAML files",
     author="Leiden University Medical Center",
@@ -39,20 +39,20 @@
     classifiers=[
         "Programming Language :: Python :: 3 :: Only",
         "Programming Language :: Python :: 3",
-        "Programming Language :: Python :: 3.6",
         "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.9",
         "Programming Language :: Python :: 3.10",
+        "Programming Language :: Python :: 3.11",
         "Development Status :: 5 - Production/Stable",
         "License :: OSI Approved :: "
         "GNU Affero General Public License v3 or later (AGPLv3+)",
         "Framework :: Pytest",
     ],
-    # Because we use the resolve(strict=False) feature from pathlib.
-    python_requires=">=3.6",
+    # Because we cannot test anymore on Python 3.6.
+    python_requires=">=3.7",
     install_requires=[
-        "pytest>=5.4.0",  # To use from_parent node instantiation.
+        "pytest>=7.0.0",  # To use pathlib Path's in pytest
         "pyyaml",
         "jsonschema"
     ],
diff --git a/tox.ini b/tox.ini
index 6b75d6c2..af66c289 100644
--- a/tox.ini
+++ b/tox.ini
@@ -6,7 +6,7 @@ envlist=py3
 deps=coverage
 commands =
     # Create HTML coverage report for humans and xml coverage report for external services.
-    coverage run --source=pytest_workflow -m py.test -v tests -m 'not functional'
+    coverage run --source=pytest_workflow -m pytest -v tests -m 'not functional'
     coverage html
     coverage xml
 
@@ -18,15 +18,12 @@ deps=flake8
      types-pyyaml
 commands =
     flake8 src tests setup.py
-    # Skip 'assert used' issue as this is a pytest plugin.'
-    # Use csv format because it has cleanest output.
-    bandit -s B101 -f csv -r src/ tests/
     mypy --ignore-missing-imports src/pytest_workflow tests/
 
 # Documentation should build on python version 3
 [testenv:docs]
 deps=-r requirements-docs.txt
-whitelist_externals=bash
+allowlist_externals=bash
                     mkdir
                     rm
 commands=
@@ -59,3 +56,10 @@ commands =
 deps=miniwdl
 commands=
     python -m pytest tests/functional/test_functional.py::test_miniwdl
+    
+[testenv:nextflow]
+# Empty deps. Otherwise deps from default testenv are used.
+deps=
+commands =
+    python -m pytest tests/functional/test_functional.py::test_nextflow
+    


Test patch:
diff --git a/docs/running_pytest_workflow.rst b/docs/running_pytest_workflow.rst
index 6bee58d7..0c6989f3 100644
--- a/docs/running_pytest_workflow.rst
+++ b/docs/running_pytest_workflow.rst
@@ -26,6 +26,20 @@ Specific pytest options for pytest workflow
     :func: __pytest_workflow_cli
     :prog: pytest
 
+Setting specific per-project settings using pytest.ini
+-----------------------------------------------------------------
+pytest can be configured `using a pytest.ini file
+<https://docs.pytest.org/en/7.1.x/reference/customize.html#pytest-ini>`_.
+This mechanic can be used to set specific settings in each repository
+where pytest workflow is used.
+
+For example a pytest.ini with the following contents::
+
+    [pytest]
+    addopts = --git-aware
+
+Can be used in a git repository with a workflow.
+
 Temporary directory cleanup and creation
 ----------------------------------------
 
@@ -115,3 +129,9 @@ Internally names and tags are handled the same so if the following tests:
       - hello
 
 are run with ``pytest --tag hello`` then both ``hello`` and ``hello2`` are run.
+
+.. note::
+
+    The pytest flags ``-k`` and ``-m`` are not supported by pytest-workflow.
+    Rational for this design decision can be `found within GitHub issue #155
+    <https://github.com/LUMC/pytest-workflow/issues/155#issuecomment-1334911457>`_.
diff --git a/docs/writing_tests.rst b/docs/writing_tests.rst
index f9b32b87..9d75d435 100644
--- a/docs/writing_tests.rst
+++ b/docs/writing_tests.rst
@@ -40,6 +40,7 @@ Test options
         must_not_contain:              # A list of strings that should NOT be in the file (optional)
           - "Cock a doodle doo"
         md5sum: e583af1f8b00b53cda87ae9ead880224   # Md5sum of the file (optional)
+        encoding: UTF-8                # Encoding for the text file (optional). Defaults to system locale.
 
   - name: simple echo                  # A second workflow. Notice the starting `-` which means
     command: "echo moo"                # that workflow items are in a list. You can add as much workflows as you want
@@ -51,6 +52,7 @@ Test options
         - "moo"
       must_not_contain:                # List of strings that should NOT be in stout (optional)
         - "Cock a doodle doo"
+      encoding: ASCII                  # Encoding for stdout (optional). Defaults to system locale.
 
   - name: mission impossible           # Also failing workflows can be tested
     tags:                              # A list of tags that can be used to select which test
@@ -60,13 +62,14 @@ Test options
     files:
       - path: "fail.log"               # Multiple files can be tested for each workflow
       - path: "TomCruise.txt.gz"       # Gzipped files can also be searched, provided their extension is '.gz'
-        contains: 
+        contains:
           - "starring"
     stderr:                            # Options for testing stderr (optional)
       contains:                        # A list of strings which should be in stderr (optional)
         - "BSOD error, please contact the IT crowd"
       must_not_contain:                # A list of strings which should NOT be in stderr (optional)
         - "Mission accomplished!"
+      encoding: UTF-16                 # Encoding for stderr (optional). Defaults to system locale.
 
   - name: regex tests
     command: echo Hello, world
@@ -83,13 +86,40 @@ Test options
 The above YAML file contains all the possible options for a workflow test.
 
 Please see the `Python documentation on regular expressions
-<https://docs.python.org/3.6/library/re.html>`_ to see how Python handles escape
+<https://docs.python.org/3/library/re.html>`_ to see how Python handles escape
 sequences.
 
 .. note::
     Workflow names must be unique. Pytest workflow will crash when multiple
     workflows have the same name, even if they are in different files.
 
+Environment variables
+----------------------
+Pytest-workflow runs tests in the same environment as in which the pytest
+executable was started. This means programs started in tests can use
+environnment variables. However, environment variables inside the command
+section itself are quoted by pytest-workflow using
+`shlex.quote <https://docs.python.org/3/library/shlex.html#shlex.quote>`_.
+See the examples below:
+
+.. code-block:: YAML
+
+    - name: Try to use an environment variable
+      command: echo $MY_VAR
+      # Output will be literally "$MY_VAR"
+
+    - name: Circumenvent shlex quoting by explicitly starting the command in a shell.
+      command: bash -c 'echo $MY_VAR'
+      # Output will be the content of $MY_VAR
+
+    - name: Use a program that checks an environment variable
+      command: singularity run my_container.sif
+      # Correctly uses "SINGULARITY_" prefixed variables
+
+If you want to use shell scripting features such as environment
+variables inside ``command``, you need to explicitly set the shell as shown
+above.
+
 Writing custom tests
 --------------------
 
diff --git a/src/pytest_workflow/content_tests.py b/src/pytest_workflow/content_tests.py
index 589e6045..27701886 100644
--- a/src/pytest_workflow/content_tests.py
+++ b/src/pytest_workflow/content_tests.py
@@ -33,72 +33,41 @@
 
 
 def check_content(strings: Iterable[str],
-                  text_lines: Iterable[str]) -> Set[str]:
+                  patterns: Iterable[str],
+                  text_lines: Iterable[str]):
     """
-    Checks whether any of the strings is present in the text lines
+    Checks whether any of the strings or patterns is present in the text lines
     It only reads the lines once and it stops reading when
-    everything is found. This makes searching for strings in large bodies of
-    text more efficient.
-    :param strings: A list of strings for which the present is checked
+    everything is found. This makes searching for strings  and patterns in
+    large bodies of text more efficient.
+    :param strings: A list of strings to check for
+    :param patterns: A list of regex patterns to check for
     :param text_lines: The lines of text that need to be searched.
-    :return: A tuple with a set of found strings, and a set of not found
-    strings
+    :return: A tuple with a set of found strings, and a set of found patterns.
     """
-
-    # Create two sets. By default all strings are not found.
     strings_to_check = set(strings)
     found_strings: Set[str] = set()
+    regex_to_match: Set[re.Pattern] = {re.compile(pattern)
+                                       for pattern in patterns}
+    found_regexes: Set[re.Pattern] = set()
 
     for line in text_lines:
         # Break the loop if all strings are found
-        # Python implements fast set equality checking by checking length first
-        if found_strings == strings_to_check:
+        if not strings_to_check and not regex_to_match:
             break
 
         for string in strings_to_check:
             if string in line:
                 found_strings.add(string)
-        # Remove found strings for faster searching. This should be done
-        # outside of the loop above.
-        strings_to_check -= found_strings
-    return found_strings
-
-
-def check_regex_content(patterns: Iterable[str],
-                        text_lines: Iterable[str]) -> Set[str]:
-    """
-    Checks whether any of the patterns is present in the text lines
-    It only reads the lines once and it stops reading when
-    everything is found. This makes searching for patterns in large bodies of
-    text more efficient.
-    :param patterns: A list of regexes which is matched
-    :param text_lines: The lines of text that need to be searched.
-    :return: A tuple with a set of found regexes, and a set of not found
-    regexes
-    """
-
-    # Create two sets. By default all strings are not found.
-    regex_to_match = {re.compile(pattern) for pattern in patterns}
-    found_patterns: Set[str] = set()
-
-    for line in text_lines:
-        # Break the loop if all regexes have been matched
-        if not regex_to_match:
-            break
-
-        # Regexes we don't have to check anymore
-        to_remove = list()
         for regex in regex_to_match:
             if re.search(regex, line):
-                found_patterns.add(regex.pattern)
-                to_remove.append(regex)
+                found_regexes.add(regex)
 
-        # Remove found patterns for faster searching. This should be done
+        # Remove found strings for faster searching. This should be done
         # outside of the loop above.
-        for regex in to_remove:
-            regex_to_match.remove(regex)
-
-    return found_patterns
+        strings_to_check -= found_strings
+        regex_to_match -= found_regexes
+    return found_strings, {x.pattern for x in found_regexes}
 
 
 class ContentTestCollector(pytest.Collector):
@@ -145,13 +114,10 @@ def find_strings(self):
                      self.filepath.open)
         try:
             # Use 'rt' here explicitly as opposed to 'rb'
-            with file_open(mode='rt') as file_handler:  # type: ignore  # mypy goes crazy here otherwise  # noqa: E501
-                self.found_strings = check_content(
+            with file_open(mode='rt', encoding=self.content_test.encoding) \
+                    as file_handler:  # type: ignore  # mypy goes crazy here otherwise  # noqa: E501
+                self.found_strings, self.found_patterns = check_content(
                     strings=strings_to_check,
-                    text_lines=file_handler)
-            # Read the file again for the regex
-            with file_open(mode='rt') as file_handler:  # type: ignore  # mypy goes crazy here otherwise  # noqa: E501
-                self.found_patterns = check_regex_content(
                     patterns=patterns_to_check,
                     text_lines=file_handler)
         except FileNotFoundError:
@@ -244,7 +210,12 @@ def runtest(self):
         were we are looking for multiple words (variants / sequences). """
         # Wait for thread to complete.
         self.parent.thread.join()
-        assert not self.parent.file_not_found
+        if not self.parent.workflow.matching_exitcode():
+            pytest.skip(f"'{self.parent.workflow.name}' did not exit with"
+                        f"desired exit code.")
+        if self.parent.file_not_found:
+            pytest.skip(f"'{self.content_name}' was not found so cannot be "
+                        f"searched")
         if self.regex:
             assert ((self.string in self.parent.found_patterns) ==
                     self.should_contain)
@@ -253,17 +224,9 @@ def runtest(self):
                     self.should_contain)
 
     def repr_failure(self, excinfo, style=None):
-        if self.parent.file_not_found:
-            containing = ("containing" if self.should_contain else
-                          "not containing")
-            return (
-                f"'{self.content_name}' does not exist and cannot be searched "
-                f"for {containing} '{self.string}'."
-            )
-        else:
-            found = "not found" if self.should_contain else "found"
-            should = "should" if self.should_contain else "should not"
-            return (
-                f"'{self.string}' was {found} in {self.content_name} "
-                f"while it {should} be there."
-            )
+        found = "not found" if self.should_contain else "found"
+        should = "should" if self.should_contain else "should not"
+        return (
+            f"'{self.string}' was {found} in {self.content_name} "
+            f"while it {should} be there."
+        )
diff --git a/src/pytest_workflow/file_tests.py b/src/pytest_workflow/file_tests.py
index 8eca7151..17642fc8 100644
--- a/src/pytest_workflow/file_tests.py
+++ b/src/pytest_workflow/file_tests.py
@@ -102,6 +102,9 @@ def runtest(self):
         # Wait for the workflow process to finish before checking if the file
         # exists.
         self.workflow.wait()
+        if not self.workflow.matching_exitcode():
+            pytest.skip(f"'{self.parent.workflow.name}' did not exit with"
+                        f"desired exit code.")
         assert self.file.exists() == self.should_exist
 
     def repr_failure(self, excinfo, style=None):
@@ -134,6 +137,9 @@ def __init__(self, parent: pytest.Collector, filepath: Path,
     def runtest(self):
         # Wait for the workflow to finish before we check the md5sum of a file.
         self.workflow.wait()
+        if not self.workflow.matching_exitcode():
+            pytest.skip(f"'{self.parent.workflow.name}' did not exit with"
+                        f"desired exit code.")
         self.observed_md5sum = file_md5sum(self.filepath)
         assert self.observed_md5sum == self.expected_md5sum
 
diff --git a/src/pytest_workflow/plugin.py b/src/pytest_workflow/plugin.py
index 5e792030..85cb148c 100644
--- a/src/pytest_workflow/plugin.py
+++ b/src/pytest_workflow/plugin.py
@@ -16,17 +16,13 @@
 
 """core functionality of pytest-workflow plugin"""
 import argparse
+import os
 import shutil
 import tempfile
 import warnings
 from pathlib import Path
 from typing import Any, Dict, List, Optional, Tuple
 
-from _pytest.config import Config as PytestConfig
-from _pytest.config.argparsing import Parser as PytestParser
-from _pytest.mark import Mark
-from _pytest.python import FunctionDefinition, Metafunc
-
 import pytest
 
 import yaml
@@ -38,7 +34,7 @@
 from .workflow import Workflow, WorkflowQueue
 
 
-def pytest_addoption(parser: PytestParser):
+def pytest_addoption(parser: pytest.Parser):
     parser.addoption(
         "--kwd", "--keep-workflow-wd",
         action="store_true",
@@ -72,7 +68,13 @@ def pytest_addoption(parser: PytestParser):
              "This ignores the .git directory, any untracked files and any "
              "files listed by .gitignore. "
              "Highly recommended when working in a git project.")
-
+    parser.addoption(
+        "--sb", "--stderr-bytes",
+        dest="stderr_bytes",
+        default=1000,
+        type=int,
+        help="The number of bytes to display from the stderr and "
+             "stdout on exitcode.")
     # Why `--tag <tag>` and not simply use `pytest -m <tag>`?
     # `-m` uses a "mark expression". So you have to type a piece of python
     # code instead of just supplying the tags you want. This is fine for the
@@ -114,16 +116,16 @@ def addoption(self, *args, **kwargs):
     return parser
 
 
-def pytest_collect_file(path, parent):
+def pytest_collect_file(file_path, path, parent):
     """Collection hook
     This collects the yaml files that start with "test" and end with
     .yaml or .yml"""
     if path.ext in [".yml", ".yaml"] and path.basename.startswith("test"):
-        return YamlFile.from_parent(parent, fspath=path)
+        return YamlFile.from_parent(parent, path=file_path)
     return None
 
 
-def pytest_configure(config: PytestConfig):
+def pytest_configure(config: pytest.Config):
     """This runs before tests start and adds values to the config."""
 
     #  Add marker to the config to prevent issues caused by:
@@ -173,13 +175,12 @@ def pytest_configure(config: PytestConfig):
         Path(basetemp) if basetemp is not None
         else Path(tempfile.mkdtemp(prefix="pytest_workflow_")))
 
-    rootdir = Path(str(config.rootdir))
-    # Raise an error if the workflow temporary directory of the rootdir
+    # Raise an error if the workflow temporary directory of the rootpath
     # (pytest's CWD). This will lead to infinite looping and copying.
-    if is_in_dir(workflow_temp_dir, rootdir):
+    if is_in_dir(workflow_temp_dir, config.rootpath):
         raise ValueError(f"'{workflow_temp_dir}' is a subdirectory of "
-                         f"'{rootdir}'. Please select a --basetemp that is "
-                         f"not in pytest's current working directory.")
+                         f"'{config.rootpath}'. Please select a --basetemp "
+                         f"that is not in pytest's current working directory.")
 
     setattr(config, "workflow_temp_dir", workflow_temp_dir)
 
@@ -195,7 +196,8 @@ def pytest_collection():
     print()
 
 
-def get_workflow_names_from_workflow_marker(marker: Mark) -> Tuple[Any, ...]:
+def get_workflow_names_from_workflow_marker(marker: pytest.Mark
+                                            ) -> Tuple[Any, ...]:
     if 'name' in marker.kwargs:
         raise DeprecationWarning(
             "Using pytest.mark.workflow(name='workflow name') is "
@@ -204,7 +206,7 @@ def get_workflow_names_from_workflow_marker(marker: Mark) -> Tuple[Any, ...]:
     return marker.args
 
 
-def pytest_generate_tests(metafunc: Metafunc):
+def pytest_generate_tests(metafunc: pytest.Metafunc):
     """
     This runs at the end of the collection phase. We use this hook to generate
     the workflow_dir fixtures for custom test functions.
@@ -215,8 +217,11 @@ def pytest_generate_tests(metafunc: Metafunc):
     if "workflow_dir" not in metafunc.fixturenames:
         return
 
-    definition: FunctionDefinition = metafunc.definition
-    marker: Optional[Mark] = definition.get_closest_marker(name="workflow")
+    # Technically definition is of type FunctionDefinition, but that is a
+    # subclass of Function and FunctionDefinition cannot be accessed through
+    # the API.
+    definition: pytest.Function = metafunc.definition
+    marker: Optional[pytest.Mark] = definition.get_closest_marker("workflow")
     if marker is None:
         raise ValueError("workflow_dir can only be requested in tests marked"
                          " with the workflow mark.")
@@ -233,12 +238,12 @@ def pytest_generate_tests(metafunc: Metafunc):
                          ids=workflow_names)
 
 
-def pytest_collection_modifyitems(config: PytestConfig,
+def pytest_collection_modifyitems(config: pytest.Config,
                                   items: List[pytest.Function]):
     """Here we skip all tests related to workflows that are not executed"""
 
     for item in items:
-        marker = item.get_closest_marker(name="workflow")
+        marker: Optional[pytest.Mark] = item.get_closest_marker("workflow")
 
         if marker is None:
             continue
@@ -401,7 +406,8 @@ def queue_workflow(self):
         # Create a workflow and make sure it runs in the tempdir
         workflow = Workflow(command=self.workflow_test.command,
                             cwd=tempdir,
-                            name=self.workflow_test.name)
+                            name=self.workflow_test.name,
+                            desired_exit_code=self.workflow_test.exit_code)
 
         # Add the workflow to the workflow queue.
         self.config.workflow_queue.put(workflow)
@@ -441,16 +447,16 @@ def collect(self):
         # Below structure makes it easy to append tests
         tests = []
 
+        tests += [ExitCodeTest.from_parent(
+            parent=self,
+            workflow=workflow,
+            stderr_bytes=self.config.getoption("stderr_bytes"))]
+
         tests += [
             FileTestCollector.from_parent(
                 parent=self, filetest=filetest, workflow=workflow)
             for filetest in self.workflow_test.files]
 
-        tests += [ExitCodeTest.from_parent(
-            parent=self,
-            desired_exit_code=self.workflow_test.exit_code,
-            workflow=workflow)]
-
         tests += [ContentTestCollector.from_parent(
             name="stdout", parent=self,
             filepath=workflow.stdout_file,
@@ -470,19 +476,29 @@ def collect(self):
 
 class ExitCodeTest(pytest.Item):
     def __init__(self, parent: pytest.Collector,
-                 desired_exit_code: int,
-                 workflow: Workflow):
-        name = f"exit code should be {desired_exit_code}"
+                 workflow: Workflow, stderr_bytes: int):
+        name = f"exit code should be {workflow.desired_exit_code}"
         super().__init__(name, parent=parent)
+        self.stderr_bytes = stderr_bytes
         self.workflow = workflow
-        self.desired_exit_code = desired_exit_code
 
     def runtest(self):
         # workflow.exit_code waits for workflow to finish.
-        assert self.workflow.exit_code == self.desired_exit_code
+        assert self.workflow.matching_exitcode()
 
     def repr_failure(self, excinfo, style=None):
-        message = (f"'{self.workflow.name}' exited with exit code " +
-                   f"'{self.workflow.exit_code}' instead of "
-                   f"'{self.desired_exit_code}'.")
+        standerr = self.workflow.stderr_file
+        standout = self.workflow.stdout_file
+        with open(standout, "rb") as standout_file, \
+             open(standerr, "rb") as standerr_file:
+            if os.path.getsize(standerr) >= self.stderr_bytes:
+                standerr_file.seek(-self.stderr_bytes, os.SEEK_END)
+            if os.path.getsize(standout) >= self.stderr_bytes:
+                standout_file.seek(-self.stderr_bytes, os.SEEK_END)
+            message = (f"'{self.workflow.name}' exited with exit code " +
+                       f"'{self.workflow.exit_code}' instead of "
+                       f"'{self.workflow.desired_exit_code}'.\nstderr: "
+                       f"{standerr_file.read().strip().decode('utf-8')}"
+                       f"\nstdout: "
+                       f"{standout_file.read().strip().decode('utf-8')}")
         return message
diff --git a/src/pytest_workflow/schema.py b/src/pytest_workflow/schema.py
index df8084a8..c8c3e40f 100644
--- a/src/pytest_workflow/schema.py
+++ b/src/pytest_workflow/schema.py
@@ -113,11 +113,13 @@ class ContentTest(object):
     def __init__(self, contains: Optional[List[str]] = None,
                  must_not_contain: Optional[List[str]] = None,
                  contains_regex: Optional[List[str]] = None,
-                 must_not_contain_regex: Optional[List[str]] = None):
+                 must_not_contain_regex: Optional[List[str]] = None,
+                 encoding: Optional[str] = None):
         self.contains: List[str] = contains or []
         self.must_not_contain: List[str] = must_not_contain or []
         self.contains_regex: List[str] = contains_regex or []
         self.must_not_contain_regex: List[str] = must_not_contain_regex or []
+        self.encoding: Optional[str] = encoding
 
 
 class FileTest(ContentTest):
@@ -127,7 +129,8 @@ def __init__(self, path: str, md5sum: Optional[str] = None,
                  contains: Optional[List[str]] = None,
                  must_not_contain: Optional[List[str]] = None,
                  contains_regex: Optional[List[str]] = None,
-                 must_not_contain_regex: Optional[List[str]] = None):
+                 must_not_contain_regex: Optional[List[str]] = None,
+                 encoding: Optional[str] = None):
         """
         A container object
         :param path: the path to the file
@@ -143,7 +146,8 @@ def __init__(self, path: str, md5sum: Optional[str] = None,
         """
         super().__init__(contains=contains, must_not_contain=must_not_contain,
                          contains_regex=contains_regex,
-                         must_not_contain_regex=must_not_contain_regex)
+                         must_not_contain_regex=must_not_contain_regex,
+                         encoding=encoding)
         self.path = Path(path)
         self.md5sum = md5sum
         self.should_exist = should_exist
diff --git a/src/pytest_workflow/schema/schema.json b/src/pytest_workflow/schema/schema.json
index c2356818..9ead66bb 100644
--- a/src/pytest_workflow/schema/schema.json
+++ b/src/pytest_workflow/schema/schema.json
@@ -67,6 +67,9 @@
             "items": {
               "type": "string"
             }
+          },
+          "encoding": {
+              "type": "string"
           }
         },
         "additionalProperties": false
@@ -97,6 +100,9 @@
             "items": {
               "type": "string"
             }
+          },
+          "encoding": {
+              "type": "string"
           }
         },
         "additionalProperties": false
@@ -140,6 +146,9 @@
               "items": {
                 "type": "string"
               }
+            },
+            "encoding": {
+              "type": "string"
             }
           },
           "required": [
diff --git a/src/pytest_workflow/util.py b/src/pytest_workflow/util.py
index 947eb71a..211e3ab3 100644
--- a/src/pytest_workflow/util.py
+++ b/src/pytest_workflow/util.py
@@ -3,7 +3,7 @@
 import os
 import re
 import shutil
-import subprocess  # nosec
+import subprocess
 import sys
 import warnings
 from pathlib import Path
@@ -48,9 +48,9 @@ def is_in_dir(child: Path, parent: Path, strict: bool = False) -> bool:
     return False
 
 
-def _run_command(*args):
+def _run_command(*args) -> str:
     """Run an external command and return the output"""
-    result = subprocess.run(args,  # nosec
+    result = subprocess.run(args,
                             stdout=subprocess.PIPE,
                             # Encoding to output as a string.
                             encoding=sys.getdefaultencoding(),
@@ -64,6 +64,19 @@ def git_root(path: Filepath) -> str:
     return output.strip()  # Remove trailing newline
 
 
+def git_check_submodules_cloned(path: Filepath):
+    output = _run_command("git", "-C", os.fspath(path), "submodule", "status",
+                          "--recursive")
+    for line in output.splitlines():
+        commit, path = line.strip().split(maxsplit=1)
+        if commit.startswith("-"):
+            raise RuntimeError(
+                f"Git submodule '{path}' was not cloned. Pytest-workflow "
+                f"cannot copy paths from non-existing submodules. Please "
+                f"clone all submodules using 'git submodule update --init "
+                f"--recursive'.")
+
+
 def git_ls_files(path: Filepath) -> List[str]:
     output = _run_command("git", "-C", os.fspath(path), "ls-files",
                           # Make sure submodules are included.
@@ -72,16 +85,16 @@ def git_ls_files(path: Filepath) -> List[str]:
     return output.strip("\n").split("\n")
 
 
-def _duplicate_tree(src: Filepath, dest: Filepath
-                    ) -> Iterator[Tuple[str, str, bool]]:
+def _recurse_directory_tree(src: Filepath, dest: Filepath
+                            ) -> Iterator[Tuple[str, str, bool]]:
     """Traverses src and for each file or directory yields a path to it,
     its destination, and whether it is a directory."""
-    for entry in os.scandir(src):  # type: os.DirEntry
+    for entry in os.scandir(src):  # type: os.DirEntry  # type: ignore
         if entry.is_dir():
             dir_src = entry.path
             dir_dest = os.path.join(dest, entry.name)
             yield dir_src, dir_dest, True
-            yield from _duplicate_tree(dir_src, dir_dest)
+            yield from _recurse_directory_tree(dir_src, dir_dest)
         elif entry.is_file() or entry.is_symlink():
             yield entry.path, os.path.join(dest, entry.name), False
         else:
@@ -89,14 +102,15 @@ def _duplicate_tree(src: Filepath, dest: Filepath
                           f"Skipping {entry.path}")
 
 
-def _duplicate_git_tree(src: Filepath, dest: Filepath
-                        ) -> Iterator[Tuple[str, str, bool]]:
+def _recurse_git_repository_tree(src: Filepath, dest: Filepath
+                                 ) -> Iterator[Tuple[str, str, bool]]:
     """Traverses src, finds all files registered in git and for each file or
     directory yields a path to it, its destination and whether it is a
     directory"""
     # A set of dirs we have already yielded. '' is the output of
     # os.path.dirname when the path is in the current directory.
     yielded_dirs: Set[str] = {''}
+    git_check_submodules_cloned(src)
     for path in git_ls_files(src):
         # git ls-files does not list directories. Yield parent first to prevent
         # creating files in non-existing directories. Also check if it is
@@ -121,6 +135,14 @@ def _duplicate_git_tree(src: Filepath, dest: Filepath
 
         # Yield the actual file if the directory has already been yielded.
         src_path = os.path.join(src, path)
+        if not os.path.exists(src_path):
+            raise FileNotFoundError(
+                f"{path} from git repository {src} is checked in in git, "
+                f"but not present in the filesystem. If the file was removed, "
+                f"its removal can be recorded in git with "
+                f"\"git -C '{src}' rm '{path}'\". "
+                f"Removal can be reversed with "
+                f"\"git -C '{src}' checkout '{path}'\".")
         dest_path = os.path.join(dest, path)
         yield src_path, dest_path, False
 
@@ -144,14 +166,16 @@ def duplicate_tree(src: Filepath, dest: Filepath,
         raise NotADirectoryError(f"Not a directory: '{src}'")
 
     if git_aware:
-        path_iter = _duplicate_git_tree(src, dest)
+        path_iter = _recurse_git_repository_tree(src, dest)
     else:
-        path_iter = _duplicate_tree(src, dest)
+        path_iter = _recurse_directory_tree(src, dest)
     if symlink:
         copy: Callable[[Filepath, Filepath], None] = \
             functools.partial(os.symlink, target_is_directory=False)
     else:
-        copy = shutil.copy2  # Preserves metadata, also used by shutil.copytree
+        # shutil.copy2 preserves metadata, also used by shutil.copytree
+        # follow_symlinks False to directly copy links
+        copy = functools.partial(shutil.copy2, follow_symlinks=False)
 
     os.makedirs(dest, exist_ok=False)
     for src_path, dest_path, is_dir in path_iter:
@@ -180,7 +204,7 @@ def file_md5sum(filepath: Path, block_size=64 * 1024) -> str:
     :param block_size: Block size in bytes
     :return: a md5sum as hexadecimal string.
     """
-    hasher = hashlib.md5()  # nosec: only used for file integrity
+    hasher = hashlib.md5()
     with filepath.open('rb') as file_handler:  # Read the file in bytes
         for block in iter(lambda: file_handler.read(block_size), b''):
             hasher.update(block)
diff --git a/src/pytest_workflow/workflow.py b/src/pytest_workflow/workflow.py
index fbc9a63d..748b0244 100644
--- a/src/pytest_workflow/workflow.py
+++ b/src/pytest_workflow/workflow.py
@@ -22,7 +22,7 @@
 """
 import queue
 import shlex
-import subprocess  # nosec: security implications have been considered
+import subprocess
 import tempfile
 import threading
 import time
@@ -35,7 +35,8 @@ class Workflow(object):
     def __init__(self,
                  command: str,
                  cwd: Optional[Path] = None,
-                 name: Optional[str] = None):
+                 name: Optional[str] = None,
+                 desired_exit_code: int = 0):
         """
         Initiates a workflow object
         :param command: The string that represents the command to be run
@@ -69,6 +70,7 @@ def __init__(self,
         self._started = False
         self.errors: List[Exception] = []
         self.start_lock = threading.Lock()
+        self.desired_exit_code = desired_exit_code
 
     def start(self):
         """Runs the workflow in a subprocess in the background.
@@ -81,7 +83,7 @@ def start(self):
                     stdout_h = self.stdout_file.open('wb')
                     stderr_h = self.stderr_file.open('wb')
                     sub_process_args = shlex.split(self.command)
-                    self._popen = subprocess.Popen(  # nosec: Shell is not enabled. # noqa
+                    self._popen = subprocess.Popen(
                         sub_process_args, stdout=stdout_h,
                         stderr=stderr_h, cwd=str(self.cwd))
                 except Exception as error:
@@ -137,6 +139,12 @@ def wait(self, timeout_secs: Optional[float] = None,
             # workflow
             pass
 
+    def matching_exitcode(self) -> bool:
+        """Checks if the workflow exited with the desired exit code"""
+        # This is done in the workflow object to reduce redundancy in the rest
+        # of the code.
+        return self.exit_code == self.desired_exit_code
+
     @property
     def stdout(self) -> bytes:
         self.wait()
diff --git a/tests/functional/simple_nextflow_test_cases.yml b/tests/functional/simple_nextflow_test_cases.yml
new file mode 100644
index 00000000..c3c0f143
--- /dev/null
+++ b/tests/functional/simple_nextflow_test_cases.yml
@@ -0,0 +1,35 @@
+- name: simple nextflow
+  command: >-
+    nextflow run nextflow_testpipeline.nf --N_LINES_TO_READ=10 --outdir results
+  files:
+    - path: results/rand/rand_0.txt
+    - path: results/rand/rand_1.txt
+    - path: results/rand/rand_2.txt
+    - path: results/rand/rand_3.txt
+    - path: results/rand/rand_4.txt
+    - path: results/rand/rand_5.txt
+    - path: results/rand/rand_6.txt
+    - path: results/rand/rand_7.txt
+    - path: results/rand/rand_8.txt
+    - path: results/rand/rand_9.txt
+    - path: results/b64/b64_0.txt
+    - path: results/b64/b64_1.txt
+    - path: results/b64/b64_2.txt
+    - path: results/b64/b64_3.txt
+    - path: results/b64/b64_4.txt
+    - path: results/b64/b64_5.txt
+    - path: results/b64/b64_6.txt
+    - path: results/b64/b64_7.txt
+    - path: results/b64/b64_8.txt
+    - path: results/b64/b64_9.txt
+    - path: results/randgz/randgz_0.txt.gz
+    - path: results/randgz/randgz_1.txt.gz
+    - path: results/randgz/randgz_2.txt.gz
+    - path: results/randgz/randgz_3.txt.gz
+    - path: results/randgz/randgz_4.txt.gz
+    - path: results/randgz/randgz_5.txt.gz
+    - path: results/randgz/randgz_6.txt.gz
+    - path: results/randgz/randgz_7.txt.gz
+    - path: results/randgz/randgz_8.txt.gz
+    - path: results/randgz/randgz_9.txt.gz
+    - path: results/all_data.gz
diff --git a/tests/functional/simple_snakefile_test_cases.yml b/tests/functional/simple_snakefile_test_cases.yml
index 5b7f969b..0ca2f666 100644
--- a/tests/functional/simple_snakefile_test_cases.yml
+++ b/tests/functional/simple_snakefile_test_cases.yml
@@ -3,19 +3,19 @@
 - name: test-config-missing
   command: snakemake -n -r -p -s SimpleSnakefile
   exit_code: 1
-  stderr:
+  stdout:
     contains:
       - "You must set --config N_LINES_TO_READ=<a value>."
 - name: test-config-wrong-type
   command: snakemake -n -r -p -s SimpleSnakefile --config N_LINES_TO_READ=one
   exit_code: 1
-  stderr:
+  stdout:
     contains:
       - "N_LINES_TO_READ must be an integer."
 - name: test-config-invalid-value
   command: snakemake -n -r -p -s SimpleSnakefile --config N_LINES_TO_READ=-1
   exit_code: 1
-  stderr:
+  stdout:
     contains:
       - "N_LINES_TO_READ must at least be 1."
 - name: test-snakemake-run
diff --git a/tests/functional/test_functional.py b/tests/functional/test_functional.py
index bf77bfa9..44c0cff2 100644
--- a/tests/functional/test_functional.py
+++ b/tests/functional/test_functional.py
@@ -19,41 +19,52 @@
 
 SIMPLE_WDL_YAML = Path(__file__).parent / "simple_wdl_test_cases.yml"
 SNAKEFILE_YAML = Path(__file__).parent / "simple_snakefile_test_cases.yml"
+NEXTFLOW_YAML = Path(__file__).parent / "simple_nextflow_test_cases.yml"
 PIPELINE_DIR = Path(__file__).parent.parent / "pipelines"
 SIMPLE_WDL = Path(PIPELINE_DIR, "wdl", "simple.wdl")
 SIMPLE_WDL_JSON = Path(PIPELINE_DIR, "wdl", "simple.json")
 SIMPLE_WDL_OPTIONS_JSON = Path(PIPELINE_DIR, "wdl", "simple.options.json")
 SNAKEFILE = Path(PIPELINE_DIR, "snakemake", "SimpleSnakefile")
+NEXTFLOWFILE = Path(PIPELINE_DIR, "nextflow", "nextflow_testpipeline.nf")
 
 
 @pytest.mark.functional
-def test_cromwell(testdir):
-    testdir.makefile(ext=".json", simple=SIMPLE_WDL_JSON.read_text())
-    testdir.makefile(ext=".wdl", simple=SIMPLE_WDL.read_text())
-    testdir.makefile(ext=".yml", test_wdl=SIMPLE_WDL_YAML.read_text())
-    testdir.makefile(ext=".options.json",
-                     simple=SIMPLE_WDL_OPTIONS_JSON.read_text())
-    result = testdir.runpytest("-v", "--tag", "cromwell",
-                               "--keep-workflow-wd-on-fail")
+def test_cromwell(pytester):
+    pytester.makefile(ext=".json", simple=SIMPLE_WDL_JSON.read_text())
+    pytester.makefile(ext=".wdl", simple=SIMPLE_WDL.read_text())
+    pytester.makefile(ext=".yml", test_wdl=SIMPLE_WDL_YAML.read_text())
+    pytester.makefile(ext=".options.json",
+                      simple=SIMPLE_WDL_OPTIONS_JSON.read_text())
+    result = pytester.runpytest("-v", "--tag", "cromwell",
+                                "--keep-workflow-wd-on-fail")
     exit_code = result.ret
     assert exit_code == 0
     assert "simple wdl" in result.stdout.str()
 
 
 @pytest.mark.functional
-def test_miniwdl(testdir):
-    testdir.makefile(ext=".json", simple=SIMPLE_WDL_JSON.read_text())
-    testdir.makefile(ext=".wdl", simple=SIMPLE_WDL.read_text())
-    testdir.makefile(ext=".yml", test_wdl=SIMPLE_WDL_YAML.read_text())
-    result = testdir.runpytest("-v", "--tag", "miniwdl",
-                               "--keep-workflow-wd-on-fail")
+def test_miniwdl(pytester):
+    pytester.makefile(ext=".json", simple=SIMPLE_WDL_JSON.read_text())
+    pytester.makefile(ext=".wdl", simple=SIMPLE_WDL.read_text())
+    pytester.makefile(ext=".yml", test_wdl=SIMPLE_WDL_YAML.read_text())
+    result = pytester.runpytest("-v", "--tag", "miniwdl",
+                                "--keep-workflow-wd-on-fail")
     assert result.ret == 0
 
 
 @pytest.mark.functional
-def test_snakemake(testdir):
-    testdir.makefile(ext="", SimpleSnakefile=SNAKEFILE.read_text())
-    testdir.makefile(ext=".yml", test_snakemake=SNAKEFILE_YAML.read_text())
-    result = testdir.runpytest("-v", "--keep-workflow-wd-on-fail")
+def test_snakemake(pytester):
+    pytester.makefile(ext="", SimpleSnakefile=SNAKEFILE.read_text())
+    pytester.makefile(ext=".yml", test_snakemake=SNAKEFILE_YAML.read_text())
+    result = pytester.runpytest("-v", "--keep-workflow-wd-on-fail")
     exit_code = result.ret
     assert exit_code == 0
+
+
+@pytest.mark.functional
+def test_nextflow(pytester):
+    pytester.makefile(ext=".nf",
+                      nextflow_testpipeline=NEXTFLOWFILE.read_text())
+    pytester.makefile(ext=".yml", test_nextflow=NEXTFLOW_YAML.read_text())
+    result = pytester.runpytest("-v", "--keep-workflow-wd-on-fail")
+    assert result.ret == 0
diff --git a/tests/pipelines/nextflow/nextflow_testpipeline.nf b/tests/pipelines/nextflow/nextflow_testpipeline.nf
new file mode 100644
index 00000000..20381250
--- /dev/null
+++ b/tests/pipelines/nextflow/nextflow_testpipeline.nf
@@ -0,0 +1,92 @@
+#!/usr/bin/env nextflow
+
+# Copyright (C) 2018 Leiden University Medical Center
+# This file is part of pytest-workflow
+#
+# pytest-workflow is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# pytest-workflow is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with pytest-workflow.  If not, see <https://www.gnu.org/licenses/
+
+# Nextflow using the Snakemake test file as example.
+# Just a simple dummy pipeline that reads some data from /dev/urandom,
+# and does some transformations on it.
+
+params.N_LINES_TO_READ = 5
+
+process read_random {
+	publishDir = [
+                path: { "${params.outdir}/rand'}
+        ]
+		
+	input:
+	val iter
+	output: 
+	val iter
+	path "rand_${iter}.txt"
+	script:
+	"""
+	head -n ${params.N_LINES_TO_READ} /dev/urandom > rand_${iter}.txt
+	"""
+}
+
+process base64_random {
+	publishDir = [
+                path: { "${params.outdir}/b64'}
+        ]
+
+	input:
+	val iter
+	path "rand_${iter}.txt"
+	output:
+	val iter
+	path "b64_${iter}.txt"
+	script:
+	"""
+	cat rand_${iter}.txt | base64 > b64_${iter}.txt
+	"""
+}
+
+process gzip_b64 {
+	publishDir = [
+                path: { "${params.outdir}/randgz'}
+        ]
+
+	input:
+	val iter
+	path "b64_${iter}.txt"
+	output:
+	path "randgz_${iter}.txt.gz"
+	script:
+	"""
+	cat b64_${iter}.txt | gzip -c > randgz_${iter}.txt.gz
+	"""
+}
+
+process concat_gzip {
+	publishDir = [
+                path: { "${params.outdir}'}
+        ]
+
+	input:
+	path 'x'
+	output:
+	path "all_data.gz"
+	script:
+	"""
+	cat ${x} > all_data.gz
+	"""
+}
+
+workflow {
+	iterations = channel.from(0..9) | read_random | base64_random | gzip_b64
+	gzip_b64.out.collect() | concat_gzip
+}
diff --git a/tests/test_content_functions.py b/tests/test_content_functions.py
index 743d4d4c..67858be3 100644
--- a/tests/test_content_functions.py
+++ b/tests/test_content_functions.py
@@ -20,7 +20,7 @@
 
 import pytest
 
-from pytest_workflow.content_tests import check_content, check_regex_content
+from pytest_workflow.content_tests import check_content
 
 LICENSE = Path(__file__).parent / "content_files" / "LICENSE"
 LICENSE_ZIPPED = LICENSE.parent / "LICENSE.gz"
@@ -48,8 +48,8 @@
 def test_check_content_succeeding(contains_strings, does_not_contain_strings):
     all_strings = set(contains_strings).union(set(does_not_contain_strings))
     with LICENSE.open("rt") as license_h:
-        found_strings = check_content(list(all_strings),
-                                      license_h)
+        found_strings, _ = check_content(
+            list(all_strings), [], license_h)
     assert set(contains_strings) == found_strings
     assert set(does_not_contain_strings) == all_strings - found_strings
 
@@ -60,8 +60,8 @@ def test_check_regex_content_succeeding(contains_regex,
                                         does_not_contain_regex):
     all_regex = set(contains_regex).union(set(does_not_contain_regex))
     with LICENSE.open("rt") as license_h:
-        found_regex = check_regex_content(list(all_regex),
-                                          license_h)
+        _, found_regex = check_content(
+            [], list(all_regex), license_h)
     assert set(contains_regex) == found_regex
     assert set(does_not_contain_regex) == all_regex - found_regex
 
@@ -72,5 +72,5 @@ def test_multiple_finds_one_line():
         "the true meaning of its creed: \"We hold these truths to be",
         "self-evident: that all men are created equal.\""]
     contains = ["dream", "day", "nation", "creed", "truths"]
-    found_strings = check_content(contains, content)
+    found_strings, _ = check_content(contains, [], content)
     assert set(contains) == found_strings
diff --git a/tests/test_encodings.py b/tests/test_encodings.py
new file mode 100644
index 00000000..03f90469
--- /dev/null
+++ b/tests/test_encodings.py
@@ -0,0 +1,44 @@
+# Copyright (C) 2018 Leiden University Medical Center
+# This file is part of pytest-workflow
+#
+# pytest-workflow is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# pytest-workflow is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with pytest-workflow.  If not, see <https://www.gnu.org/licenses/
+
+"""Check how different encodings are handled by pytest-workflow"""
+
+import textwrap
+TEST_TEXT = """In het Nederlands komen verscheidene diakritische tekens voor.
+Een veel gebruikte is het trema ofwel de dubbele puntjes die boven een letter
+geplaatst worden vaak wanneer een meervoud een onduidelijke klank zou hebben
+zonder de verduidelijking van het trema. Zoals bijvoorbeeld: zee -> zeeën,
+bacterie -> bacteriën.
+Daarnaast worden veel diakritische tekens gebruikt in leenworden. Denk hierbij
+aan woorden als: überhaupt, crème fraîche en Curaçao."""
+
+
+def test_encoding(pytester):
+    pytester.makefile(".yml", textwrap.dedent("""
+    - name: test_encoding
+      command: "bash -c 'true'"
+      files:
+        - path: diakritische_tekens.txt
+          encoding: UTF32
+          contains:
+            - überhaupt
+            - crème fraîche
+    """))
+    test_txt = pytester.path / "diakritische_tekens.txt"
+    # UTF32 is not the default on windows and linux I believe
+    test_txt.write_text(TEST_TEXT, encoding="UTF32")
+    result = pytester.runpytest("-v")
+    assert result.ret == 0
diff --git a/tests/test_fail_messages.py b/tests/test_fail_messages.py
index bd373e0e..4640bd06 100644
--- a/tests/test_fail_messages.py
+++ b/tests/test_fail_messages.py
@@ -29,6 +29,11 @@
     """,
      "'fail_test' exited with exit code '2' instead of '0'"),
     ("""\
+    - name: exitcode_test
+      command: bash -c 'printf "This code had an error" >&2 ; exit 12'
+    """,
+     "stderr: This code had an error"),
+    ("""\
     - name: create file
       command: echo moo
       files:
@@ -89,7 +94,7 @@
      "it should not be there"),
     ("""\
     - name: fail_test
-      command: grep
+      command: bash -c 'echo "" >&2'
       stderr:
         contains:
           - "No arguments?"
@@ -97,34 +102,15 @@
      "'No arguments?' was not found in 'fail_test': stderr "
      "while it should be there"),
     ("""\
-    - name: fail_test
-      command: grep
-      stderr:
+   - name: fail_test
+     command: bash -c 'echo "grep --help" >&2'
+     stderr:
         must_not_contain:
           - "grep --help"
     """,
      "'grep --help' was found in 'fail_test': stderr while "
      "it should not be there"),
     ("""\
-    - name: file_not_exist
-      command: echo moo
-      files:
-        - path: moo.txt
-          contains:
-            - "moo"
-    """,
-     "moo.txt' does not exist and cannot be searched for containing 'moo'."),
-    ("""\
-    - name: file_not_exist
-      command: echo moo
-      files:
-        - path: moo.txt
-          must_not_contain:
-            - "miaow"
-    """,
-     "moo.txt' does not exist and cannot be searched for "
-     "not containing 'miaow'."),
-    ("""\
     - name: simple echo
       command: "echo Hello, world"
       stdout:
@@ -164,9 +150,32 @@
 
 
 @pytest.mark.parametrize(["test", "message"], FAILURE_MESSAGE_TESTS)
-def test_messages(test: str, message: str, testdir):
-    testdir.makefile(".yml", textwrap.dedent(test))
+def test_messages(test: str, message: str, pytester):
+    pytester.makefile(".yml", textwrap.dedent(test))
+    # Ideally this should be run in a LC_ALL=C environment. But this is not
+    # possible due to multiple levels of process launching.
+    result = pytester.runpytest("-v")
+    assert message in result.stdout.str()
+
+
+EXITCODE_MESSAGE_TESTS: List[Tuple[str, str]] = [
+    ("""\
+    - name: stderr_bytes_stderr_test
+      command: bash -c 'printf "This code had an error" ; exit 12'
+    """,
+     "stdout: error"),
+    ("""\
+    - name: stderr_bytes_stdout_test
+      command: bash -c 'printf "This code had an error" >&2 ; exit 12'
+    """,
+     "stderr: error")
+]
+
+
+@pytest.mark.parametrize(["test", "message"], EXITCODE_MESSAGE_TESTS)
+def test_messages_exitcode(test: str, message: str, pytester):
+    pytester.makefile(".yml", textwrap.dedent(test))
     # Ideally this should be run in a LC_ALL=C environment. But this is not
     # possible due to multiple levels of process launching.
-    result = testdir.runpytest("-v")
+    result = pytester.runpytest("-v", "--sb", "5")
     assert message in result.stdout.str()
diff --git a/tests/test_miscellaneous_crashes.py b/tests/test_miscellaneous_crashes.py
index 0c47809a..4c78f47e 100644
--- a/tests/test_miscellaneous_crashes.py
+++ b/tests/test_miscellaneous_crashes.py
@@ -17,10 +17,10 @@
 from .test_success_messages import SIMPLE_ECHO
 
 
-def test_same_name_different_files(testdir):
-    testdir.makefile(".yml", test_a=SIMPLE_ECHO)
-    testdir.makefile(".yml", test_b=SIMPLE_ECHO)
-    result = testdir.runpytest()
+def test_same_name_different_files(pytester):
+    pytester.makefile(".yml", test_a=SIMPLE_ECHO)
+    pytester.makefile(".yml", test_b=SIMPLE_ECHO)
+    result = pytester.runpytest()
     assert result.ret != 0
     assert ("Workflow name 'simple echo' used more than once"
             in result.stdout.str())
diff --git a/tests/test_multithreading.py b/tests/test_multithreading.py
index 91fee6ed..51498800 100644
--- a/tests/test_multithreading.py
+++ b/tests/test_multithreading.py
@@ -36,10 +36,10 @@
 
 
 @pytest.mark.parametrize(["threads"], [(1,), (2,), (4,)])
-def test_multithreaded(threads, testdir):
+def test_multithreaded(threads, pytester):
     test = MULTHITHREADED_TEST
     test_number = len(test)
-    testdir.makefile(".yml", test=yaml.safe_dump(test))
+    pytester.makefile(".yml", test=yaml.safe_dump(test))
 
     # Calculate how many iterations are needed to process all the tests
     # For example: 4 tests with 2 threads. 2 can be finished simultaneously
@@ -49,7 +49,7 @@ def test_multithreaded(threads, testdir):
                   else test_number // threads + 1)
 
     start_time = time.time()
-    testdir.runpytest("-v", "--wt", str(threads))
+    pytester.runpytest("-v", "--wt", str(threads))
     end_time = time.time()
     completion_time = end_time - start_time
     # If the completion time is shorter than (iterations * SLEEP_TIME), too
diff --git a/tests/test_skip_tests.py b/tests/test_skip_tests.py
new file mode 100644
index 00000000..d95e0f2b
--- /dev/null
+++ b/tests/test_skip_tests.py
@@ -0,0 +1,70 @@
+# Copyright (C) 2018 Leiden University Medical Center
+# This file is part of pytest-workflow
+#
+# pytest-workflow is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# pytest-workflow is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with pytest-workflow.  If not, see <https://www.gnu.org/licenses/
+
+import textwrap
+
+SKIP_TESTS = textwrap.dedent("""\
+- name: wall_test
+  command: echo "testing"
+  files:
+    - path: test.txt
+      contains:
+        - "brie"
+    - path: test2.txt
+      contains:
+        - "halloumi"
+      must_not_contain:
+        - "gorgonzola"
+""")
+SKIP_TESTS2 = textwrap.dedent("""
+- name: wall2_test
+  command: bash -c "echo 'testing' > test3.txt"
+  files:
+    - path: test3.txt
+      contains:
+        - "kaas"
+      must_not_contain:
+        - "testing"
+""")
+SKIP_TESTS3 = textwrap.dedent("""
+- name: wall3_test
+  command: bash -c "exit 12"
+  files:
+    - path: test4.txt
+      contains:
+        - "content"
+      must_not_contain:
+        - "no_content"
+      md5sum: e583af1f8b00b53cda87ae9ead880224
+""")
+
+
+def test_skips(pytester):
+    pytester.makefile(".yml", test=SKIP_TESTS)
+    result = pytester.runpytest("-v").parseoutcomes()
+    assert {"failed": 2, "passed": 1, "skipped": 3} == result
+
+
+def test_skips2(pytester):
+    pytester.makefile(".yml", test=SKIP_TESTS2)
+    result = pytester.runpytest("-v").parseoutcomes()
+    assert {"failed": 2, "passed": 2} == result
+
+
+def test_skips3(pytester):
+    pytester.makefile(".yml", test=SKIP_TESTS3)
+    result = pytester.runpytest("-v").parseoutcomes()
+    assert {"failed": 1, "skipped": 4} == result
diff --git a/tests/test_success_messages.py b/tests/test_success_messages.py
index 566e486b..8157c8dd 100644
--- a/tests/test_success_messages.py
+++ b/tests/test_success_messages.py
@@ -16,12 +16,10 @@
 """Tests the success messages"""
 
 import shutil
-import subprocess  # nosec
+import subprocess
 import textwrap
 from pathlib import Path
 
-from _pytest.tmpdir import TempdirFactory
-
 import pytest
 
 MOO_FILE = textwrap.dedent("""\
@@ -111,17 +109,17 @@
 
 
 @pytest.fixture(scope="session")
-def succeeding_tests_output(tmpdir_factory: TempdirFactory):
-    """This fixture was written because the testdir function has a default
+def succeeding_tests_output(tmp_path_factory: pytest.TempPathFactory):
+    """This fixture was written because the pytester function has a default
     scope of 'function'. This is very inefficient when testing multiple
     success messages in the output as the whole test yaml with all commands
     has to be run again.
     This fixture runs the succeeding tests once with pytest -v"""
-    tempdir = str(tmpdir_factory.mktemp("succeeding_tests"))
+    tempdir = tmp_path_factory.mktemp("succeeding_tests")
     test_file = Path(tempdir, "test_succeeding.yml")
     with test_file.open("w") as file_handler:
         file_handler.write(SUCCEEDING_TESTS_YAML)
-    process_out = subprocess.run(args=["pytest", "-v"],  # nosec
+    process_out = subprocess.run(args=["pytest", "-v"],
                                  stdout=subprocess.PIPE,
                                  stderr=subprocess.PIPE,
                                  cwd=tempdir)
@@ -139,8 +137,8 @@ def test_message_success_no_errors_or_fails(succeeding_tests_output):
     assert "FAIL" not in succeeding_tests_output
 
 
-def test_message_directory_kept_no_errors_or_fails(testdir):
-    testdir.makefile(".yml", test=SUCCEEDING_TESTS_YAML)
-    result = testdir.runpytest("-v", "--keep-workflow-wd")
+def test_message_directory_kept_no_errors_or_fails(pytester):
+    pytester.makefile(".yml", test=SUCCEEDING_TESTS_YAML)
+    result = pytester.runpytest("-v", "--keep-workflow-wd")
     assert "ERROR" not in result.stdout.str()
     assert "FAIL" not in result.stdout.str()
diff --git a/tests/test_tags.py b/tests/test_tags.py
index b37b4cd8..aa1282ce 100644
--- a/tests/test_tags.py
+++ b/tests/test_tags.py
@@ -39,35 +39,35 @@
 """)
 
 
-def test_name_tag_with_space(testdir):
-    testdir.makefile(".yml", test_tags=TAG_TESTS)
-    result = testdir.runpytest("-v", "--tag", "three again").stdout.str()
+def test_name_tag_with_space(pytester):
+    pytester.makefile(".yml", test_tags=TAG_TESTS)
+    result = pytester.runpytest("-v", "--tag", "three again").stdout.str()
     assert "three again" in result
     assert "four" not in result
     assert "nine" not in result
 
 
-def test_name_tag(testdir):
-    testdir.makefile(".yml", test_tags=TAG_TESTS)
-    result = testdir.runpytest("-v", "--tag", "three").stdout.str()
+def test_name_tag(pytester):
+    pytester.makefile(".yml", test_tags=TAG_TESTS)
+    result = pytester.runpytest("-v", "--tag", "three").stdout.str()
     assert "three" in result
     assert "three again" not in result
     assert "four" not in result
     assert "nine" not in result
 
 
-def test_category_tag(testdir):
-    testdir.makefile(".yml", test_tags=TAG_TESTS)
-    result = testdir.runpytest("-v", "--tag", "odd").stdout.str()
+def test_category_tag(pytester):
+    pytester.makefile(".yml", test_tags=TAG_TESTS)
+    result = pytester.runpytest("-v", "--tag", "odd").stdout.str()
     assert "three" in result
     assert "three again" in result
     assert "nine" in result
     assert "four" not in result
 
 
-def test_category_tag2(testdir):
-    testdir.makefile(".yml", test_tags=TAG_TESTS)
-    result = testdir.runpytest("-v", "--tag", "even").stdout.str()
+def test_category_tag2(pytester):
+    pytester.makefile(".yml", test_tags=TAG_TESTS)
+    result = pytester.runpytest("-v", "--tag", "even").stdout.str()
     assert "three" not in result
     assert "three again" not in result
     assert "four" in result
diff --git a/tests/test_temp_directory.py b/tests/test_temp_directory.py
index 85319c50..3aa45e0d 100644
--- a/tests/test_temp_directory.py
+++ b/tests/test_temp_directory.py
@@ -21,23 +21,26 @@
 import textwrap
 from pathlib import Path
 
+import pytest
+
 from .test_success_messages import SIMPLE_ECHO
 
 
-def test_directory_kept(testdir):
-    testdir.makefile(".yml", test=SIMPLE_ECHO)
-    result = testdir.runpytest("-v", "--keep-workflow-wd")
-    working_dir = re.search(r"command:   echo moo\n\tdirectory: ([\w/_-]*)",
-                            result.stdout.str()).group(1)
+def test_directory_kept(pytester: pytest.Pytester):
+    pytester.makefile(".yml", test=SIMPLE_ECHO)
+    result = pytester.runpytest("-v", "--keep-workflow-wd")
+    match = re.search(r"command:   echo moo\n\tdirectory: ([\w/_-]*)",
+                      result.stdout.str())
+    assert match is not None
+    working_dir = match.group(1)
     assert Path(working_dir).exists()
     assert Path(working_dir, "log.out").exists()
     assert Path(working_dir, "log.err").exists()
-    shutil.rmtree(str(testdir))
 
 
-def test_directory_not_kept(testdir):
-    testdir.makefile(".yml", test=SIMPLE_ECHO)
-    result = testdir.runpytest("-v")
+def test_directory_not_kept(pytester):
+    pytester.makefile(".yml", test=SIMPLE_ECHO)
+    result = pytester.runpytest("-v")
     working_dir = re.search(r"command:   echo moo\n\tdirectory: ([\w/_-]*)",
                             result.stdout.str()).group(1)
     assert not Path(working_dir).exists()
@@ -46,10 +49,10 @@ def test_directory_not_kept(testdir):
             ) in result.stdout.str()
 
 
-def test_basetemp_correct(testdir):
-    testdir.makefile(".yml", test=SIMPLE_ECHO)
+def test_basetemp_correct(pytester):
+    pytester.makefile(".yml", test=SIMPLE_ECHO)
     tempdir = tempfile.mkdtemp()
-    result = testdir.runpytest("-v", "--basetemp", tempdir)
+    result = pytester.runpytest("-v", "--basetemp", tempdir)
     message = (f"\tcommand:   echo moo\n"
                f"\tdirectory: {tempdir}/simple_echo\n"
                f"\tstdout:    {tempdir}/simple_echo/log.out\n"
@@ -57,16 +60,16 @@ def test_basetemp_correct(testdir):
     assert message in result.stdout.str()
 
 
-def test_basetemp_can_be_used_twice(testdir):
-    testdir.makefile(".yml", test=SIMPLE_ECHO)
+def test_basetemp_can_be_used_twice(pytester):
+    pytester.makefile(".yml", test=SIMPLE_ECHO)
     tempdir = tempfile.mkdtemp()
     # First run to fill up tempdir
-    testdir.runpytest("-v", "--keep-workflow-wd", "--basetemp", tempdir)
+    pytester.runpytest("-v", "--keep-workflow-wd", "--basetemp", tempdir)
     # Make sure directory is there.
     assert Path(tempdir, "simple_echo").exists()
     # Run again with same basetemp.
-    result = testdir.runpytest("-v", "--keep-workflow-wd", "--basetemp",
-                               tempdir)
+    result = pytester.runpytest("-v", "--keep-workflow-wd", "--basetemp",
+                                tempdir)
     exit_code = result.ret
     assert (f"'{tempdir}/simple_echo' already exists. Deleting ..." in
             result.stdout.str())
@@ -74,27 +77,27 @@ def test_basetemp_can_be_used_twice(testdir):
     shutil.rmtree(tempdir)
 
 
-def test_basetemp_will_be_created(testdir):
-    testdir.makefile(".yml", test=SIMPLE_ECHO)
+def test_basetemp_will_be_created(pytester):
+    pytester.makefile(".yml", test=SIMPLE_ECHO)
     # This creates an empty dir
     tempdir_base = tempfile.mkdtemp()
     # This path should not exist
     tempdir = Path(tempdir_base, "non", "existing")
     # If pytest-workflow does not handle non-existing nested directories well
     # it should crash.
-    result = testdir.runpytest("-v", "--keep-workflow-wd", "--basetemp",
-                               str(tempdir))
+    result = pytester.runpytest("-v", "--keep-workflow-wd", "--basetemp",
+                                str(tempdir))
     assert tempdir.exists()
     assert result.ret == 0
     shutil.rmtree(tempdir_base)
 
 
-def test_basetemp_can_not_be_in_rootdir(testdir):
-    testdir.makefile(".yml", test=SIMPLE_ECHO)
-    testdir_path = Path(str(testdir.tmpdir))
-    tempdir = testdir_path / "tmp"
-    result = testdir.runpytest("-v", "--basetemp", str(tempdir))
-    message = f"'{str(tempdir)}' is a subdirectory of '{str(testdir_path)}'"
+def test_basetemp_can_not_be_in_rootdir(pytester: pytest.Pytester):
+    pytester.makefile(".yml", test=SIMPLE_ECHO)
+    pytester.makefile(".yml", test=SIMPLE_ECHO)
+    tempdir = pytester.path / "tmp"
+    result = pytester.runpytest("-v", "--basetemp", str(tempdir))
+    message = f"'{str(tempdir)}' is a subdirectory of '{str(pytester.path)}'"
     assert message in result.stderr.str()
 
 
@@ -109,9 +112,9 @@ def test_basetemp_can_not_be_in_rootdir(testdir):
 """
 
 
-def test_directory_kept_on_fail(testdir):
-    testdir.makefile(".yml", test=FAIL_TEST)
-    result = testdir.runpytest("-v", "--keep-workflow-wd-on-fail")
+def test_directory_kept_on_fail(pytester):
+    pytester.makefile(".yml", test=FAIL_TEST)
+    result = pytester.runpytest("-v", "--keep-workflow-wd-on-fail")
     working_dir = re.search(
         r"command:   bash -c 'exit 1'\n\tdirectory: ([\w/_-]*)",
         result.stdout.str()).group(1)
@@ -123,9 +126,9 @@ def test_directory_kept_on_fail(testdir):
     shutil.rmtree(working_dir)
 
 
-def test_directory_not_kept_on_succes(testdir):
-    testdir.makefile(".yml", test=SUCCESS_TEST)
-    result = testdir.runpytest("-v", "--kwdof")
+def test_directory_not_kept_on_succes(pytester):
+    pytester.makefile(".yml", test=SUCCESS_TEST)
+    result = pytester.runpytest("-v", "--kwdof")
     working_dir = re.search(
         r"command:   bash -c 'exit 0'\n\tdirectory: ([\w/_-]*)",
         result.stdout.str()).group(1)
@@ -134,11 +137,11 @@ def test_directory_not_kept_on_succes(testdir):
             result.stdout.str())
 
 
-def test_directory_of_symlinks(testdir):
-    testdir.makefile(".yml", test=SIMPLE_ECHO)
-    subdir = testdir.mkdir("subdir")
+def test_directory_of_symlinks(pytester):
+    pytester.makefile(".yml", test=SIMPLE_ECHO)
+    subdir = pytester.mkdir("subdir")
     Path(str(subdir), "subfile.txt").write_text("test")
-    result = testdir.runpytest("-v", "--symlink", "--kwd")
+    result = pytester.runpytest("-v", "--symlink", "--kwd")
     working_dir = re.search(
         r"command:   echo moo\n\tdirectory: ([\w/_-]*)",
         result.stdout.str()).group(1)
@@ -148,7 +151,7 @@ def test_directory_of_symlinks(testdir):
     shutil.rmtree(working_dir)
 
 
-def test_directory_unremovable_message(testdir):
+def test_directory_unremovable_message(pytester):
     # Following directory contains nested contents owned by root.
     test = textwrap.dedent("""
     - name: Create unremovable dir
@@ -156,8 +159,8 @@ def test_directory_unremovable_message(testdir):
         bash -c "docker run -v $(pwd):$(pwd) -w $(pwd) debian \
         bash -c 'mkdir test && touch test/test'"
     """)
-    testdir.makefile(".yaml", test=test)
-    result = testdir.runpytest()
+    pytester.makefile(".yaml", test=test)
+    result = pytester.runpytest()
     assert ("Unable to remove the following directories due to permission "
             "errors" in result.stdout.str())
     assert result.ret == 0
diff --git a/tests/test_threading_errors.py b/tests/test_threading_errors.py
index 860a440d..910f96cb 100644
--- a/tests/test_threading_errors.py
+++ b/tests/test_threading_errors.py
@@ -19,13 +19,13 @@
 import textwrap
 
 
-def test_shlex_error(testdir):
+def test_shlex_error(pytester):
     test = textwrap.dedent("""\
     - name: wrong command
       command: a command with a dangling double-quote"
     """)
-    testdir.makefile(".yml", test=test)
-    result = testdir.runpytest("-v")
+    pytester.makefile(".yml", test=test)
+    result = pytester.runpytest("-v")
     assert "shlex" in result.stdout.str()
     assert "ValueError: No closing quotation" in result.stdout.str()
     assert "'wrong command' python error during start" in result.stdout.str()
diff --git a/tests/test_utils.py b/tests/test_utils.py
index aa74946e..005d5a72 100644
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -16,13 +16,14 @@
 import hashlib
 import os
 import shutil
-import subprocess  # nosec
+import subprocess
 import tempfile
 from pathlib import Path
 
 import pytest
 
-from pytest_workflow.util import duplicate_tree, file_md5sum, git_root, \
+from pytest_workflow.util import duplicate_tree, file_md5sum, \
+    git_check_submodules_cloned, git_root, \
     is_in_dir, link_tree, replace_whitespace
 
 WHITESPACE_TESTS = [
@@ -100,9 +101,9 @@ def git_dir():
     (git_dir / "test").mkdir()
     test_file = git_dir / "test" / "test.txt"
     test_file.touch()
-    subprocess.run(["git", "-C", str(git_dir), "init"])  # nosec
-    subprocess.run(["git", "-C", str(git_dir), "add", str(test_file)])  # nosec
-    subprocess.run(["git", "-C", str(git_dir), "commit", "-m",  # nosec
+    subprocess.run(["git", "-C", str(git_dir), "init"])
+    subprocess.run(["git", "-C", str(git_dir), "add", str(test_file)])
+    subprocess.run(["git", "-C", str(git_dir), "commit", "-m",
                     "initial commit"])
     yield git_dir
     shutil.rmtree(git_dir)
@@ -117,6 +118,16 @@ def test_duplicate_git_tree(git_dir):
     assert (dest / "test" / "test.txt").exists()
 
 
+def test_duplicate_git_tree_file_removed_error(git_dir):
+    assert (git_dir / ".git").exists()
+    os.remove(git_dir / "test" / "test.txt")
+    dest = Path(tempfile.mkdtemp()) / "test"
+    with pytest.raises(FileNotFoundError) as e:
+        duplicate_tree(git_dir, dest, git_aware=True)
+    e.match("checked in")
+    e.match(f"\"git -C '{git_dir}' rm '{str(Path('test', 'test.txt'))}'\"")
+
+
 def test_duplicate(git_dir):
     assert (git_dir / ".git").exists()
     dest = Path(tempfile.mkdtemp()) / "test"
@@ -145,7 +156,74 @@ def test_git_root(git_dir):
 
 @pytest.mark.parametrize("hash_file", HASH_FILE_DIR.iterdir())
 def test_file_md5sum(hash_file: Path):
-    # No sec added because this hash is only used for checking file integrity
-    whole_file_md5 = hashlib.md5(hash_file.read_bytes()).hexdigest()  # nosec
+    whole_file_md5 = hashlib.md5(hash_file.read_bytes()).hexdigest()
     per_line_md5 = file_md5sum(hash_file)
     assert whole_file_md5 == per_line_md5
+
+
+def create_git_repo(path):
+    dir = Path(path)
+    os.mkdir(dir)
+    file = dir / "README.md"
+    file.write_text("# My new project\n\nHello this project is awesome!\n")
+    subdir = dir / "sub"
+    subdir.mkdir()
+    another_file = subdir / "subtext.md"
+    another_file.write_text("# Subtext\n\nSome other example text.\n")
+    subdir_link = dir / "gosub"
+    subdir_link.symlink_to(subdir.relative_to(dir), target_is_directory=True)
+    subprocess.run(["git", "init", "-b", "main"], cwd=dir)
+    subprocess.run(["git", "config", "user.name", "A U Thor"], cwd=dir)
+    subprocess.run(["git", "config", "user.email", "author@example.com"],
+                   cwd=dir)
+    subprocess.run(["git", "add", "."], cwd=dir)
+    subprocess.run(["git", "commit", "-m", "initial commit"], cwd=dir)
+
+
+@pytest.fixture()
+def git_repo_with_submodules():
+    repo_dir = Path(tempfile.mkdtemp())
+    bird_repo = repo_dir / "bird"
+    nest_repo = repo_dir / "nest"
+    create_git_repo(bird_repo)
+    create_git_repo(nest_repo)
+    # https://bugs.launchpad.net/ubuntu/+source/git/+bug/1993586
+    subprocess.run(["git", "-c", "protocol.file.allow=always",
+                    "submodule", "add", bird_repo.absolute()],
+                   cwd=nest_repo.absolute())
+    subprocess.run(["git", "commit", "-m", "add bird repo as a submodule"],
+                   cwd=nest_repo.absolute())
+    yield nest_repo
+    shutil.rmtree(repo_dir)
+
+
+def test_git_submodule_check(git_repo_with_submodules, tmp_path):
+    cloned_repo = tmp_path / "cloned"
+    subprocess.run(
+        # No recursive clone
+        ["git", "-c", "protocol.file.allow=always",
+         "clone", git_repo_with_submodules.absolute(), cloned_repo.absolute()],
+        cwd=tmp_path
+    )
+    with pytest.raises(RuntimeError) as error:
+        git_check_submodules_cloned(cloned_repo)
+    # Error message should allow user to resolve the issue.
+    error.match("'git submodule update --init --recursive'")
+    subprocess.run(["git", "-c", "protocol.file.allow=always", "submodule",
+                    "update", "--init", "--recursive"],
+                   cwd=cloned_repo.absolute())
+    # Check error does not occur when issue resolved.
+    git_check_submodules_cloned(cloned_repo)
+
+
+# https://github.com/LUMC/pytest-workflow/issues/162
+def test_duplicate_git_tree_submodule_symlinks(git_repo_with_submodules):
+    assert (git_repo_with_submodules / ".git").exists()
+    dest = Path(tempfile.mkdtemp()) / "test"
+    duplicate_tree(git_repo_with_submodules, dest, git_aware=True)
+    assert dest.exists()
+    assert not (dest / ".git").exists()
+    link = dest / "bird" / "gosub"
+    assert link.exists()
+    assert link.is_symlink()
+    assert link.resolve() == dest / "bird" / "sub"
diff --git a/tests/test_warnings.py b/tests/test_warnings.py
index 24589f95..3b68f36b 100644
--- a/tests/test_warnings.py
+++ b/tests/test_warnings.py
@@ -22,20 +22,20 @@
 
 # Add a test to make sure no deprecation, or other warnings occur due to
 # changes in upstream libraries.
-def test_no_warnings(testdir):
+def test_no_warnings(pytester):
     basetemp = tempfile.mkdtemp()
-    testdir.makefile(".yml", test_a=MOO_FILE)
-    result = testdir.runpytest("--basetemp", basetemp)
+    pytester.makefile(".yml", test_a=MOO_FILE)
+    result = pytester.runpytest("--basetemp", basetemp)
     outcomes = result.parseoutcomes()
     assert outcomes.get('warnings', 0) == 0
     shutil.rmtree(basetemp)
 
 
-def test_git_warning(testdir):
+def test_git_warning(pytester):
     basetemp = tempfile.mkdtemp()
-    testdir.mkdir(".git")
-    testdir.makefile(".yml", test_a=MOO_FILE)
-    result = testdir.runpytest("--basetemp", basetemp)
+    pytester.mkdir(".git")
+    pytester.makefile(".yml", test_a=MOO_FILE)
+    result = pytester.runpytest("--basetemp", basetemp)
     outcomes = result.parseoutcomes()
     assert outcomes.get('warnings') == 1
     assert ".git dir detected" in result.stdout.str()
diff --git a/tests/test_workflow.py b/tests/test_workflow.py
index 83b637c5..46344278 100644
--- a/tests/test_workflow.py
+++ b/tests/test_workflow.py
@@ -15,7 +15,7 @@
 # along with pytest-workflow.  If not, see <https://www.gnu.org/licenses/
 
 """Tests the Workflow class"""
-import subprocess  # nosec. This is just for the Timeout exception.
+import subprocess
 
 import pytest
 
@@ -96,3 +96,12 @@ def test_workflow_name():
 def test_workflow_name_inferred():
     workflow = Workflow("echo moo")
     assert workflow.name == "echo"
+
+
+def test_workflow_matching_exit_code():
+    workflow = Workflow("echo moo")
+    workflow.run()
+    assert workflow.matching_exitcode()
+    workflow2 = Workflow("grep", desired_exit_code=2)
+    workflow2.run()
+    assert workflow2.matching_exitcode()
diff --git a/tests/test_workflow_dependent_tests.py b/tests/test_workflow_dependent_tests.py
index 48959a23..66711297 100644
--- a/tests/test_workflow_dependent_tests.py
+++ b/tests/test_workflow_dependent_tests.py
@@ -40,28 +40,28 @@ def test_hook_impl():
 
 
 @pytest.mark.parametrize("test", [TEST_HOOK_ARGS])
-def test_not_skipped(test, testdir):
-    testdir.makefile(".yml", test_simple=SIMPLE_ECHO)
-    testdir.makefile(".py", test_hook=test)
-    result = testdir.runpytest()
+def test_not_skipped(test, pytester):
+    pytester.makefile(".yml", test_simple=SIMPLE_ECHO)
+    pytester.makefile(".py", test_hook=test)
+    result = pytester.runpytest()
     result.assert_outcomes(passed=5)
 
 
-def test_name_use_is_deprecated(testdir):
-    testdir.makefile(".py", test_hook=TEST_HOOK_KWARGS)
-    testdir.makefile(".yml", test_simple=SIMPLE_ECHO)
-    result = testdir.runpytest().stdout.str()
+def test_name_use_is_deprecated(pytester):
+    pytester.makefile(".py", test_hook=TEST_HOOK_KWARGS)
+    pytester.makefile(".yml", test_simple=SIMPLE_ECHO)
+    result = pytester.runpytest().stdout.str()
     assert "Use pytest.mark.workflow('workflow_name') instead." in result
     assert "DeprecationWarning" in result
 
 
 @pytest.mark.parametrize("test", [TEST_HOOK_ARGS])
-def test_skipped(test, testdir):
-    testdir.makefile(".yml", test_simple=SIMPLE_ECHO)
-    testdir.makefile(".py", test_hook=test)
+def test_skipped(test, pytester):
+    pytester.makefile(".yml", test_simple=SIMPLE_ECHO)
+    pytester.makefile(".py", test_hook=test)
     # No workflow will run because the tag does not match
     # ``-r s`` gives the reason why a test was skipped.
-    result = testdir.runpytest("-v", "-r", "s", "--tag", "flaksdlkad")
+    result = pytester.runpytest("-v", "-r", "s", "--tag", "flaksdlkad")
     result.assert_outcomes(skipped=1)
     assert "'simple echo' has not run." in result.stdout.str()
 
@@ -77,30 +77,30 @@ def test_fixture_impl(workflow_dir):
 
 
 @pytest.mark.parametrize("test", [TEST_FIXTURE_ARGS])
-def test_workflow_dir_arg(test, testdir):
+def test_workflow_dir_arg(test, pytester):
     # Call the test, `test_asimple` because tests are run alphabetically.
     # This will detect if the workflow dir has been removed.
-    testdir.makefile(".yml", test_asimple=SIMPLE_ECHO)
-    testdir.makefile(".py", test_fixture=test)
-    result = testdir.runpytest()
+    pytester.makefile(".yml", test_asimple=SIMPLE_ECHO)
+    pytester.makefile(".py", test_fixture=test)
+    result = pytester.runpytest()
     result.assert_outcomes(passed=5, failed=0, errors=0, skipped=0)
 
 
 @pytest.mark.parametrize("test", [TEST_FIXTURE_ARGS])
-def test_workflow_dir_arg_skipped(test, testdir):
+def test_workflow_dir_arg_skipped(test, pytester):
     """Run this test to check if this does not run into fixture request
     errors"""
-    testdir.makefile(".yml", test_asimple=SIMPLE_ECHO)
-    testdir.makefile(".py", test_fixture=test)
-    result = testdir.runpytest("-v", "-r", "s", "--tag", "flaksdlkad")
+    pytester.makefile(".yml", test_asimple=SIMPLE_ECHO)
+    pytester.makefile(".py", test_fixture=test)
+    result = pytester.runpytest("-v", "-r", "s", "--tag", "flaksdlkad")
     result.assert_outcomes(skipped=1)
 
 
 @pytest.mark.parametrize("test", [TEST_FIXTURE_ARGS])
-def test_mark_not_unknown(test, testdir):
-    testdir.makefile(".yml", test_asimple=SIMPLE_ECHO)
-    testdir.makefile(".py", test_fixture=test)
-    result = testdir.runpytest("-v")
+def test_mark_not_unknown(test, pytester):
+    pytester.makefile(".yml", test_asimple=SIMPLE_ECHO)
+    pytester.makefile(".py", test_fixture=test)
+    result = pytester.runpytest("-v")
     assert "PytestUnknownMarkWarning" not in result.stdout.str()
 
 
@@ -113,12 +113,12 @@ def test_fixture_impl(workflow_dir):
 """)
 
 
-def test_workflow_not_exist_dir_arg(testdir):
+def test_workflow_not_exist_dir_arg(pytester):
     """Run this test to check if this does not run into fixture request
     errors"""
-    testdir.makefile(".yml", test_asimple=SIMPLE_ECHO)
-    testdir.makefile(".py", test_fixture=TEST_FIXTURE_WORKFLOW_NOT_EXIST)
-    result = testdir.runpytest("-v", "-r", "s")
+    pytester.makefile(".yml", test_asimple=SIMPLE_ECHO)
+    pytester.makefile(".py", test_fixture=TEST_FIXTURE_WORKFLOW_NOT_EXIST)
+    result = pytester.runpytest("-v", "-r", "s")
     result.assert_outcomes(skipped=1, passed=4)
     assert "'shoobiedoewap' has not run." in result.stdout.str()
 
@@ -131,12 +131,12 @@ def test_fixture_impl(workflow_dir):
 """)
 
 
-def test_fixture_unmarked_test(testdir):
+def test_fixture_unmarked_test(pytester):
     """Run this test to check if this does not run into fixture request
     errors"""
-    testdir.makefile(".yml", test_asimple=SIMPLE_ECHO)
-    testdir.makefile(".py", test_fixture=TEST_FIXTURE_UNMARKED_TEST)
-    result = testdir.runpytest("-v", "-r", "s")
+    pytester.makefile(".yml", test_asimple=SIMPLE_ECHO)
+    pytester.makefile(".py", test_fixture=TEST_FIXTURE_UNMARKED_TEST)
+    result = pytester.runpytest("-v", "-r", "s")
     assert ("workflow_dir can only be requested in tests marked with "
             "the workflow mark.") in result.stdout.str()
 
@@ -151,12 +151,12 @@ def test_fixture_impl(workflow_dir):
 """)
 
 
-def test_mark_wrong_key_with_fixture(testdir):
+def test_mark_wrong_key_with_fixture(pytester):
     """Run this test to check if this does not run into fixture request
     errors"""
-    testdir.makefile(".yml", test_asimple=SIMPLE_ECHO)
-    testdir.makefile(".py", test_fixture=TEST_MARK_WRONG_KEY)
-    result = testdir.runpytest("-v", "-r", "s")
+    pytester.makefile(".yml", test_asimple=SIMPLE_ECHO)
+    pytester.makefile(".py", test_fixture=TEST_MARK_WRONG_KEY)
+    result = pytester.runpytest("-v", "-r", "s")
     assert ("A workflow name or names should be defined in the "
             "workflow marker of test_fixture.py::test_fixture_impl"
             ) in result.stdout.str()
@@ -164,7 +164,7 @@ def test_mark_wrong_key_with_fixture(testdir):
     result.assert_outcomes(passed=0, failed=0, skipped=0, errors=1)
 
 
-def test_fixture_usable_for_file_tests(testdir):
+def test_fixture_usable_for_file_tests(pytester):
     test_workflow = textwrap.dedent("""\
     - name: number files
       command: >-
@@ -189,13 +189,13 @@ def test_div_by_three(workflow_dir):
         assert int(number_file_content) % 3 == 0
     """)
 
-    testdir.makefile(".yml", test_aworkflow=test_workflow)
-    testdir.makefile(".py", test_div=test_div_by_three)
-    result = testdir.runpytest("-v")
+    pytester.makefile(".yml", test_aworkflow=test_workflow)
+    pytester.makefile(".py", test_div=test_div_by_three)
+    result = pytester.runpytest("-v")
     result.assert_outcomes(passed=4, failed=0, skipped=0, errors=0)
 
 
-def test_same_custom_test_multiple_times(testdir):
+def test_same_custom_test_multiple_times(pytester):
     test_workflow = textwrap.dedent("""\
     - name: one_two_three
       command: >-
@@ -226,9 +226,9 @@ def test_div_by_three(workflow_dir):
         assert int(number_file.read_text()) % 3 == 0
     """)
 
-    testdir.makefile(".yml", test_aworkflow=test_workflow)
-    testdir.makefile(".py", test_div=test_div_by_three)
-    result = testdir.runpytest("-v")
+    pytester.makefile(".yml", test_aworkflow=test_workflow)
+    pytester.makefile(".py", test_div=test_div_by_three)
+    result = pytester.runpytest("-v")
     result.assert_outcomes(passed=9, failed=0, skipped=0, errors=0)
 
 
@@ -263,19 +263,19 @@ def test_div_by_three(workflow_dir):
 """)
 
 
-def test_same_custom_test_multiple_times_one_error(testdir):
-    testdir.makefile(".yml", test_aworkflow=TEST_WORKFLOWS)
-    testdir.makefile(".py", test_div=TEST_DIV_BY_THREE)
-    result = testdir.runpytest("-v")
+def test_same_custom_test_multiple_times_one_error(pytester):
+    pytester.makefile(".yml", test_aworkflow=TEST_WORKFLOWS)
+    pytester.makefile(".py", test_div=TEST_DIV_BY_THREE)
+    result = pytester.runpytest("-v")
     result.assert_outcomes(passed=8, failed=1, skipped=0, errors=0)
     assert ("test_div.py::test_div_by_three[two_three_five] FAILED "
             in result.stdout.str())
 
 
-def test_custom_tests_properly_skipped(testdir):
-    testdir.makefile(".yml", test_aworkflow=TEST_WORKFLOWS)
-    testdir.makefile(".py", test_div=TEST_DIV_BY_THREE)
-    result = testdir.runpytest("-v", "--tag", "one_two_three")
+def test_custom_tests_properly_skipped(pytester):
+    pytester.makefile(".yml", test_aworkflow=TEST_WORKFLOWS)
+    pytester.makefile(".py", test_div=TEST_DIV_BY_THREE)
+    result = pytester.runpytest("-v", "--tag", "one_two_three")
     result.assert_outcomes(passed=3, failed=0, skipped=2, errors=0)
     assert ("test_div.py::test_div_by_three[two_three_five] SKIPPED "
             in result.stdout.str())
diff --git a/tests/yamls/valid/dream_file.yaml b/tests/yamls/valid/dream_file.yaml
index 72b9d7e2..d5096d69 100644
--- a/tests/yamls/valid/dream_file.yaml
+++ b/tests/yamls/valid/dream_file.yaml
@@ -11,16 +11,19 @@
         - "blabla"
       must_not_contain:
         - "stuff"
+      encoding: UTF8
   stdout:
     contains:
       - "bla"
     must_not_contain:
       - "not_bla"
+    encoding: ASCII
   stderr:
     contains:
       - "bla"
     must_not_contain:
       - "not_bla"
+    encoding: UTF8
   exit_code: 127
   command: "the one string"
 - name: other test