Repository: adamchainz/pytest-flake8dir (language: python)

Issue:
Return stderr in addition to stdout
I've just added a bunch of deprecation warnings to a plugin, but I have no way of testing them using this package because no information about stderr is returned. Ideally, I would be able to add tests to verify that the deprecation warnings are firing correctly.


Discussion / hints:

Fix patch:
diff --git a/HISTORY.rst b/HISTORY.rst
index 61f6a8a..dbe731a 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -2,6 +2,12 @@
 History
 =======
 
+* Add ``err`` and ``err_lines`` attributes for accessing the stderr of the
+  flake8 process.
+
+  Thanks to Matthew Gamble for the feature request in `Issue #221
+  <https://github.com/adamchainz/django-linear-migrations/issues/221>`__.
+
 2.5.0 (2021-05-10)
 ------------------
 
diff --git a/README.rst b/README.rst
index 4cad799..1ae8618 100644
--- a/README.rst
+++ b/README.rst
@@ -57,7 +57,7 @@ API
 ---------------------
 
 A pytest fixture that wraps Pytest's built-in ``tmpdir`` fixture
-(`docs <https://docs.pytest.org/en/latest/tmpdir.html>`_), to create a
+(`docs <https://docs.pytest.org/en/latest/tmpdir.html>`__), to create a
 temporary directory, allow adding files, and running flake8.
 
 If you're using this to test a flake8 plugin, make sure flake8 is picking up
@@ -168,7 +168,12 @@ Represents the parsed output of a flake8 run.
 ``Flake8Result.out: str``
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The full string of output generated by flake8.
+The full string of output (stdout) generated by flake8.
+
+``Flake8Result.err: str``
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The full string of error output (stderr) generated by flake8.
 
 ``Flake8Result.exit_code: int``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -189,4 +194,7 @@ For example, given a result you can check for a particular line being output:
     expected = "./example.py:1:2: E221 multiple spaces before operator"
     assert expected in result.out_lines
 
-.. _tmpdir: https://docs.pytest.org/en/latest/tmpdir.html
+``Flake8Result.err_lines: List[str]``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Like ``out_lines``, but for error output.


Test patch:
diff --git a/src/pytest_flake8dir.py b/src/pytest_flake8dir.py
index 1d09427..b884b18 100644
--- a/src/pytest_flake8dir.py
+++ b/src/pytest_flake8dir.py
@@ -59,15 +59,17 @@ def run_flake8(self, extra_args=None):
             universal_newlines=True,
         )
         process.wait()
-        return Flake8Result(process.stdout.read(), process.returncode)
+        return Flake8Result(
+            out=process.stdout.read(),
+            err=process.stderr.read(),
+            exit_code=process.returncode,
+        )
 
 
 class Flake8Result:
-    def __init__(self, out, exit_code):
+    def __init__(self, out, err, exit_code):
         self.out = out
+        self.out_lines = out.strip().splitlines()
+        self.err = err
+        self.err_lines = err.strip().splitlines()
         self.exit_code = exit_code
-
-        lines = out.strip().split("\n")
-        if lines[-1] == "":
-            lines = lines[:-1]
-        self.out_lines = lines
diff --git a/tests/test_pytest_flake8dir.py b/tests/test_pytest_flake8dir.py
index 4dfd4cd..95dddf8 100644
--- a/tests/test_pytest_flake8dir.py
+++ b/tests/test_pytest_flake8dir.py
@@ -89,6 +89,45 @@ def test_make_file(flake8dir):
     ]
 
 
+def test_error_output(flake8dir):
+    flake8dir.make_setup_cfg(
+        """
+        [flake8]
+        extend-select = MC1
+
+        [flake8:local-plugins]
+        extension =
+            MC1 = example:MyChecker1
+        paths = .
+        """
+    )
+    flake8dir.make_example_py(
+        """
+        import warnings
+
+        warnings.warn("This is a warning!", UserWarning)
+
+
+        class MyChecker1:
+            name = "MyChecker1"
+            version = "1"
+
+            def __init__(self, tree):
+                self.tree = tree
+
+            def run(self):
+                if False:
+                    yield
+        """
+    )
+
+    result = flake8dir.run_flake8()
+
+    assert "This is a warning!" in result.err
+    assert result.err_lines[0].endswith("UserWarning: This is a warning!")
+    assert result.err_lines[1] == '  warnings.warn("This is a warning!", UserWarning)'
+
+
 def test_extra_args(flake8dir):
     flake8dir.make_py_files(
         example="""