Repository: NiklasRosenstein/slap (language: python)

Issue:
Support  pytest xfail
I have a parametrized test with `marks=[pytest.mark.xfail]` where I expect it to fail.

However, I get an exception when running `shut pkg test --no-capture`:

```
Traceback (most recent call last):
  File "/.../bin/shut", line 8, in <module>
    sys.exit(shut())
  File "/.../lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/.../lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/.../lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/.../lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/.../lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/.../lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/.../lib/python3.9/site-packages/shut/commands/pkg/test.py", line 153, in test
    test_run = test_package(package, isolate, keep_test_env, capture)
  File "/.../lib/python3.9/site-packages/shut/commands/pkg/test.py", line 70, in test_package
    return package.test_driver.test_package(package, runtime, capture)
  File "/.../lib/python3.9/site-packages/shut/test/pytest.py", line 126, in test_package
    test_run = load_report_file(self.report_file)
  File "/.../lib/python3.9/site-packages/shut/test/pytest.py", line 81, in load_report_file
    test_status = {'passed': TestStatus.PASSED, 'failed': TestStatus.FAILED, 'skipped': TestStatus.SKIPPED}[test['outcome']]
KeyError: 'xfailed'
```

Indeed, only  `passed`, `failed` and `skipped` are acknowledged. Whereas `xfailed` and `xpassed` are not.

Documentation: https://docs.pytest.org/en/6.2.x/example/parametrize.html
Support  pytest xfail
I have a parametrized test with `marks=[pytest.mark.xfail]` where I expect it to fail.

However, I get an exception when running `shut pkg test --no-capture`:

```
Traceback (most recent call last):
  File "/.../bin/shut", line 8, in <module>
    sys.exit(shut())
  File "/.../lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/.../lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/.../lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/.../lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/.../lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/.../lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/.../lib/python3.9/site-packages/shut/commands/pkg/test.py", line 153, in test
    test_run = test_package(package, isolate, keep_test_env, capture)
  File "/.../lib/python3.9/site-packages/shut/commands/pkg/test.py", line 70, in test_package
    return package.test_driver.test_package(package, runtime, capture)
  File "/.../lib/python3.9/site-packages/shut/test/pytest.py", line 126, in test_package
    test_run = load_report_file(self.report_file)
  File "/.../lib/python3.9/site-packages/shut/test/pytest.py", line 81, in load_report_file
    test_status = {'passed': TestStatus.PASSED, 'failed': TestStatus.FAILED, 'skipped': TestStatus.SKIPPED}[test['outcome']]
KeyError: 'xfailed'
```

Indeed, only  `passed`, `failed` and `skipped` are acknowledged. Whereas `xfailed` and `xpassed` are not.

Documentation: https://docs.pytest.org/en/6.2.x/example/parametrize.html


Discussion / hints:
Thanks for reporting @Mulugruntz , I'll take a look at it soon!
Thanks for reporting @Mulugruntz , I'll take a look at it soon!

Fix patch:
diff --git a/.changelog/_unreleased.yml b/.changelog/_unreleased.yml
new file mode 100644
index 00000000..73ee689a
--- /dev/null
+++ b/.changelog/_unreleased.yml
@@ -0,0 +1,7 @@
+release_date: null
+changes:
+- type: fix
+  component: general
+  description: fix missing import for `typing.Dict` in `pytest` driver module, which
+    could in some Python versions result in a `NameError`
+  fixes: []


Test patch:
diff --git a/src/shut/test/pytest.py b/src/shut/test/pytest.py
index 65ae5406..28bd9152 100644
--- a/src/shut/test/pytest.py
+++ b/src/shut/test/pytest.py
@@ -24,7 +24,7 @@
 import os
 import subprocess as sp
 import sys
-from typing import List, Optional, Tuple, TYPE_CHECKING
+from typing import Dict, List, Optional, Tuple, TYPE_CHECKING
 
 from databind.core import datamodel, field
 from nr.parsing.date import timezone  # type: ignore
@@ -78,7 +78,7 @@ def load_report_file(report_file: str) -> TestRun:
     else:
       crash = None
       stdout = None
-    test_status = {'passed': TestStatus.PASSED, 'failed': TestStatus.FAILED, 'skipped': TestStatus.SKIPPED}[test['outcome']]
+    test_status = {'passed': TestStatus.PASSED, 'failed': TestStatus.FAILED, 'skipped': TestStatus.SKIPPED}[test['outcome'].lstrip('x')]
     tests.append(TestCase(
       name=test['nodeid'],
       duration=sum(test[k]['duration'] for k in ('setup', 'call', 'teardown') if k in test),
diff --git a/src/test/test_test.py b/src/test/test_test.py
new file mode 100644
index 00000000..2939fefa
--- /dev/null
+++ b/src/test/test_test.py
@@ -0,0 +1,11 @@
+
+"""
+This module uses some pytest features that should all work when using the pytest driver with `shut pkg test`.
+"""
+
+import pytest
+
+
+@pytest.mark.xfail
+def test_xfail():
+  pass