Repository: kiwitcms/pytest-plugin (language: python)

Issue:
Add tcms-api calls to pytest plugin.
Add tcms-api calls to pytest plugin to communicate and report test results Kiwi TCMS. 

still wip.


Discussion / hints:

Fix patch:
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
new file mode 100644
index 0000000..528daca
--- /dev/null
+++ b/.github/workflows/lint.yml
@@ -0,0 +1,20 @@
+name: lint
+
+on:
+  push:
+    branches: master
+  pull_request:
+
+jobs:
+  black:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.8]
+
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+      - uses: psf/black@stable
diff --git a/Makefile b/Makefile
index fed360e..acb73d5 100644
--- a/Makefile
+++ b/Makefile
@@ -8,11 +8,11 @@ flake8:
 
 .PHONY: pylint
 pylint:
-	pylint -d missing-docstring tcms_pytest_plugin/ tests/
+	pylint --load-plugins=pylint.extensions.no_self_use -d missing-docstring tcms_pytest_plugin/ tests/
 
 .PHONY: test
 test:
-	pytest --kiwitcms
+	PYTHONPATH=. pytest -p tcms_pytest_plugin --kiwitcms tests/
 
 .PHONY: check-build
 check-build:
diff --git a/README.rst b/README.rst
index f463e4d..b2fafac 100644
--- a/README.rst
+++ b/README.rst
@@ -10,14 +10,6 @@ kiwitcms-pytest-plugin
     :target: https://pypi.org/project/kiwitcms-pytest-plugin
     :alt: Python versions
 
-.. image:: https://travis-ci.org/kiwitcms/pytest-plugin.svg?branch=master
-    :target: https://travis-ci.org/kiwitcms/pytest-plugin
-    :alt: See Build Status on Travis CI
-
-.. image:: https://coveralls.io/repos/github/kiwitcms/pytest-plugin/badge.svg?branch=master
-    :target: https://coveralls.io/github/kiwitcms/pytest-plugin?branch=master
-    :alt: Code coverage
-
 .. image:: https://tidelift.com/badges/package/pypi/kiwitcms
     :target: https://tidelift.com/subscription/pkg/pypi-kiwitcms?utm_source=pypi-kiwitcms&utm_medium=github&utm_campaign=readme
     :alt: Tidelift
@@ -31,40 +23,52 @@ kiwitcms-pytest-plugin
     :alt: Kiwi TCMS on Twitter
 
 
-Pytest plugin for Kiwi TCMS
-
-----
+This is a `pytest`_ plugin for `Kiwi TCMS <http://kiwitcms.org>`_.
 
 
-Features
---------
+Installation
+------------
 
-* TODO
+You can install "kiwitcms-pytest-plugin" via `pip`_ from `PyPI`_::
 
+    $ pip install kiwitcms-pytest-plugin
 
-Requirements
-------------
 
-* TODO
+Configuration and environment
+-----------------------------
 
+Minimal config file `~/.tcms.conf`::
 
-Installation
-------------
+    [tcms]
+    url = https://tcms.server/xml-rpc/
+    username = your-username
+    password = your-password
 
-You can install "kiwitcms-pytest-plugin" via `pip`_ from `PyPI`_::
 
-    $ pip install kiwitcms-pytest-plugin
+For more info see `tcms-api docs <https://tcms-api.readthedocs.io>`_.
+Further documentation and behavior specification can be found
+`here <https://tcms-api.readthedocs.io/en/latest/modules/tcms_api.plugin_helpers.html>`_.
 
 
 Usage
 -----
 
-* TODO
+::
+
+    # define environment variables
+    pytest -p tcms_pytest_plugin --kiwitcms
+
+
+Changelog
+---------
+
+v0.1.0 (05 Jul 2022)
+~~~~~~~~~~~~~~~~~~~~
+
+- First release on PyPI
+- Based on tcms-api v11.3
+- Compatible with Kiwi TCMS v11.3 or later
 
-Contributing
-------------
-Contributions are very welcome. Tests can be run with `tox`_, please ensure
-the coverage at least stays the same before you submit a pull request.
 
 License
 -------
@@ -73,6 +77,14 @@ Distributed under the terms of the `GNU GPL v3.0`_ license,
 "kiwitcms-pytest-plugin" is free and open source software
 
 
+Authors
+-------
+
+- `Dmitry Dygalo <https://github.com/Stranger6667>`_
+- `Bryan Mutai <https://github.com/brymut>`_
+- `Alex Todorov <https://github.com/atodorov>`_
+
+
 Issues
 ------
 
@@ -82,6 +94,5 @@ please `file an issue`_ along with a detailed description.
 .. _`GNU GPL v3.0`: http://www.gnu.org/licenses/gpl-3.0.txt
 .. _`file an issue`: https://github.com/kiwitcms/pytest-plugin/issues
 .. _`pytest`: https://github.com/pytest-dev/pytest
-.. _`tox`: https://tox.readthedocs.io/en/latest/
 .. _`pip`: https://pypi.org/project/pip/
 .. _`PyPI`: https://pypi.org/project
diff --git a/devel.txt b/devel.txt
index 7abea3d..440fd08 100644
--- a/devel.txt
+++ b/devel.txt
@@ -1,6 +1,7 @@
 -r requirements.txt
 
 doc8
+black
 flake8
 pylint_django
 twine
diff --git a/setup.cfg b/setup.cfg
index 7da1f96..4b2d6da 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -1,2 +1,6 @@
 [flake8]
-max-line-length = 100
+max-line-length = 102
+extend-ignore = E203
+
+[pylint]
+max-line-length = 102
diff --git a/setup.py b/setup.py
index 3cb5add..ddcf54f 100644
--- a/setup.py
+++ b/setup.py
@@ -1,18 +1,37 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
+#
 # Copyright (c) 2019 Dmitry Dygalo <dadygalo@gmail.com>
+# Copyright (c) 2021 Bryan Mutai <mutaiwork@gmail.com>
+# Copyright (c) 2021-2022 Alexander Todorov <atodorov@otb.bg>
+#
 # Licensed under the GPLv3: https://www.gnu.org/licenses/gpl.html
+import os
 from setuptools import setup
-import tcms_pytest_plugin
 
-README = open("README.rst").read()
-REQUIREMENTS = open("requirements.txt").readlines()
+
+def get_version():
+    version_py_path = os.path.join("tcms_pytest_plugin", "version.py")
+    with open(version_py_path, encoding="utf-8") as version_file:
+        version = version_file.read()
+        return (
+            version.replace(" ", "")
+            .replace("__version__=", "")
+            .strip()
+            .strip("'")
+            .strip('"')
+        )
+
+
+with open("README.rst", encoding="utf-8") as readme:
+    README = readme.read()
+
+with open("requirements.txt", encoding="utf-8") as requirements:
+    REQUIREMENTS = requirements.readlines()
 
 setup(
     name="kiwitcms-pytest-plugin",
-    version=tcms_pytest_plugin.__version__,
-    author="Dmitry Dygalo",
-    author_email="dadygalo@gmail.com",
+    version=get_version(),
     maintainer="Kiwi TCMS",
     maintainer_email="info@kiwitcms.org",
     url="https://github.com/kiwitcms/pytest-plugin",
@@ -40,5 +59,5 @@
         "Topic :: Software Development :: Quality Assurance",
         "Topic :: Software Development :: Testing",
     ],
-    entry_points={"pytest11": ["tcms_pytest_plugin = tcms_pytest_plugin.plugin"]},
+    entry_points={"pytest11": ["tcms_pytest_plugin = tcms_pytest_plugin"]},
 )


Test patch:
diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
index 63a363e..85f669c 100644
--- a/.github/workflows/testing.yml
+++ b/.github/workflows/testing.yml
@@ -16,7 +16,33 @@ jobs:
       fail-fast: false
       matrix:
         python-version: [3.8]
-        make-command: [doc8, flake8, pylint, test, check-build]
+        make-command: [doc8, flake8, pylint, check-build]
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v2
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v1
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install Python dependencies
+        run: |
+          pip install -U pip setuptools
+          pip install -r devel.txt
+
+      - name: Verify ${{ matrix.make-command }}
+        run: |
+          make ${{ matrix.make-command }}
+
+  test:
+    name: test / Py${{ matrix.python-version }}
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: [3.8]
 
     steps:
       - name: Check out code
@@ -29,12 +55,10 @@ jobs:
 
       - name: Install Python dependencies
         run: |
-          sudo apt-get install virtualenv
           pip install -U pip setuptools
           pip install -r devel.txt
 
       - name: Prepare for integration test
-        if: matrix.make-command == 'test'
         run: |
           echo "[tcms]" > ~/.tcms.conf
           echo "url = https://tcms.kiwitcms.org/xml-rpc/" >> ~/.tcms.conf
@@ -43,11 +67,16 @@ jobs:
           sudo cp .ssl/*.pem /usr/lib/ssl/certs/
           sudo update-ca-certificates --fresh
 
-      - name: Verify ${{ matrix.make-command }}
+      - name: Execute tests
         run: |
           export TCMS_PRODUCT=$GITHUB_REPOSITORY
           # branch name or pull/123
           export TCMS_PRODUCT_VERSION=$(echo $GITHUB_REF | sed "s|refs/heads/||" | sed "s|refs/||" | sed "s|/merge||")
           # short commit number
           export TCMS_BUILD=$(echo $GITHUB_SHA | cut -c1-7)
-          make ${{ matrix.make-command }}
+
+          # will also report to Kiwi TCMS
+          make test | tee -a results.log
+
+          # make sure results are as expected
+          tail -n1 results.log | grep "1 failed, 1 passed, 1 skipped, 1 error"
diff --git a/tcms_pytest_plugin/__init__.py b/tcms_pytest_plugin/__init__.py
index 81f8a59..77a1301 100644
--- a/tcms_pytest_plugin/__init__.py
+++ b/tcms_pytest_plugin/__init__.py
@@ -1,3 +1,92 @@
+# -*- coding: utf-8 -*-
 # Copyright (c) 2019 Dmitry Dygalo <dadygalo@gmail.com>
+# Copyright (c) 2021 Bryan Mutai <mutaiwork@gmail.com>
+# Copyright (c) 2021-2022 Alexander Todorov <atodorov@otb.bg>
+#
 # Licensed under the GPLv3: https://www.gnu.org/licenses/gpl.html
-__version__ = "0.1.0"
+#
+# pylint: disable=unused-argument, no-self-use
+
+
+import pytest
+from tcms_api import plugin_helpers
+
+from .version import __version__
+
+
+def pytest_addoption(parser):
+    group = parser.getgroup("kiwitcms")
+    group.addoption(
+        "--kiwitcms",
+        action="store_true",
+        help="Enable recording of test results in Kiwi TCMS.",
+    )
+
+
+def pytest_configure(config):
+    if config.getoption("--kiwitcms"):
+        config.pluginmanager.register(
+            KiwiTCMSPlugin(config.getoption("--verbose")),
+            name="kiwitcms",
+        )
+
+
+class Backend(plugin_helpers.Backend):
+    name = "kiwitcms-pytest-plugin"
+    version = __version__
+
+
+class KiwiTCMSPlugin:
+    executions = []
+    status_id = 0
+    status_weight = 0
+    comment = ""
+
+    def __init__(self, verbose=False):
+        self.backend = Backend(prefix="[pytest]", verbose=verbose)
+
+    def pytest_runtestloop(self, session):
+        self.backend.configure()
+
+    def pytest_runtest_logstart(self, nodeid, location):
+        self.status_weight = 0
+        self.comment = ""
+
+        test_case, _ = self.backend.test_case_get_or_create(nodeid)
+        self.backend.add_test_case_to_plan(test_case["id"], self.backend.plan_id)
+        self.executions = self.backend.add_test_case_to_run(
+            test_case["id"], self.backend.run_id
+        )
+
+    @pytest.hookimpl(hookwrapper=True)
+    def pytest_report_teststatus(self, report, config):
+        yield
+
+        # this hooks is executed 3 times for setup/call/teardown each of which
+        # has their own status. Thus a positive status cannot override a negative one
+        # Positive statuses are considered only in the beginning
+        if report.when in ["setup", "call"]:
+            if self.status_weight >= 0 and report.outcome == "passed":
+                self.status_id = self.backend.get_status_id("PASSED")
+                self.status_weight = 1
+            elif self.status_weight >= 0 and report.outcome == "skipped":
+                self.status_id = self.backend.get_status_id("WAIVED")
+                self.status_weight = 1
+
+        # a negative status is stronger, will always override any other status
+        # negative statuses are considered in every occasion
+        if report.outcome in ["error", "failed"]:
+            self.status_id = self.backend.get_status_id(report.outcome.upper())
+            self.status_weight = -1
+            self.comment = f"<pre>\n{report.longrepr}\n</pre>"
+
+    def pytest_runtest_logfinish(self, nodeid, location):
+        for execution in self.executions:
+            self.backend.update_test_execution(
+                execution["id"], self.status_id, self.comment
+            )
+
+    @pytest.hookimpl(hookwrapper=True)
+    def pytest_terminal_summary(self, terminalreporter, exitstatus, config):
+        yield
+        self.backend.finish_test_run()
diff --git a/tcms_pytest_plugin/plugin.py b/tcms_pytest_plugin/plugin.py
deleted file mode 100644
index ca451d6..0000000
--- a/tcms_pytest_plugin/plugin.py
+++ /dev/null
@@ -1,60 +0,0 @@
-# -*- coding: utf-8 -*-
-# Copyright (c) 2019 Dmitry Dygalo <dadygalo@gmail.com>
-# Licensed under the GPLv3: https://www.gnu.org/licenses/gpl.html
-import configparser
-import os
-
-import attr
-import pytest
-
-DEFAULT_CONFIG_PATH = "~/.tcms.conf"
-
-
-def pytest_addoption(parser):
-    group = parser.getgroup("kiwitcms")
-    group.addoption(
-        "--kiwitcms", action="store_true", help="Enable recording of test results in Kiwi TCMS."
-    )
-
-
-def pytest_configure(config):
-    if config.getoption("--kiwitcms"):
-        config_manager = ConfigManager(config)
-        config.pluginmanager.register(
-            Plugin(
-                api_url=config_manager.get_option("url", "TCMS_API_URL"),
-                username=config_manager.get_option("username", "TCMS_USERNAME"),
-                password=config_manager.get_option("password", "TCMS_PASSWORD"),
-            ),
-            name="pytest-kiwitcms",
-        )
-
-
-def is_not_none(instance, attribute, value):  # pylint: disable=unused-argument
-    if not value:
-        pytest.exit(msg=f"Option {attribute.name} is empty", returncode=1)
-
-
-@attr.s(hash=True)
-class Plugin:  # pylint: disable=too-few-public-methods
-    api_url = attr.ib(type=str, validator=is_not_none)
-    username = attr.ib(type=str, validator=is_not_none)
-    password = attr.ib(type=str, validator=is_not_none)
-
-
-@attr.s
-class ConfigManager:  # pylint: disable=too-few-public-methods
-    """Helper for configuration parsing."""
-
-    pytest_config = attr.ib()
-    config_file = attr.ib(init=False)
-
-    def __attrs_post_init__(self):
-        self.config_file = configparser.ConfigParser()
-        self.config_file.read(os.path.expanduser(DEFAULT_CONFIG_PATH))
-
-    def get_option(self, config_name, envvar):
-        try:
-            return self.config_file["tcms"][config_name]
-        except KeyError:
-            return os.getenv(envvar)
diff --git a/tcms_pytest_plugin/version.py b/tcms_pytest_plugin/version.py
new file mode 100644
index 0000000..3dc1f76
--- /dev/null
+++ b/tcms_pytest_plugin/version.py
@@ -0,0 +1,1 @@
+__version__ = "0.1.0"
diff --git a/tests/conftest.py b/tests/conftest.py
index 70dd6dd..3d0abf2 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -2,7 +2,7 @@
 # Licensed under the GPLv3: https://www.gnu.org/licenses/gpl.html
 import pytest
 
-pytest_plugins = "pytester"  # pylint: disable=invalid-name
+pytest_plugins = ["pytester"]  # pylint: disable=invalid-name
 
 
 DEFAULT_CONFIG_CONTENT = """
diff --git a/tests/test_kiwitcms.py b/tests/test_kiwitcms.py
new file mode 100644
index 0000000..21d4fd7
--- /dev/null
+++ b/tests/test_kiwitcms.py
@@ -0,0 +1,24 @@
+# -*- coding: utf-8 -*-
+# Copyright (c) 2019 Dmitry Dygalo <dadygalo@gmail.com>
+# Copyright (c) 2022 Alexander Todorov <atodorov@otb.bg>
+# Licensed under the GPLv3: https://www.gnu.org/licenses/gpl.html
+#
+# pylint: disable=comparison-of-constants, unused-argument
+import pytest
+
+
+def test_should_pass_when_assertion_passes():
+    assert True
+
+
+def test_should_fail_when_assertion_fails():
+    assert 1 == 2
+
+
+@pytest.mark.skip("We've decided not to test this")
+def test_should_skip_if_marked_as_such():
+    assert True
+
+
+def test_should_error_with_non_existing_fixture(non_existing):
+    assert 1 == ""
diff --git a/tests/test_pytest_kiwitcms.py b/tests/test_pytest_kiwitcms.py
deleted file mode 100644
index 01fd367..0000000
--- a/tests/test_pytest_kiwitcms.py
+++ /dev/null
@@ -1,96 +0,0 @@
-# -*- coding: utf-8 -*-
-# Copyright (c) 2019 Dmitry Dygalo <dadygalo@gmail.com>
-# Licensed under the GPLv3: https://www.gnu.org/licenses/gpl.html
-import pytest
-
-
-DEFAULT_CONFIG = (
-    ("TCMS_API_URL", "http://another.com"),
-    ("TCMS_USERNAME", "user"),
-    ("TCMS_PASSWORD", "random"),
-)
-
-
-@pytest.mark.usefixtures("config_file")
-def test_file_configuration(testdir):
-    """Load configuration from ~/.tcms.conf."""
-    testdir.makepyfile(
-        """
-        def test_file_configuration(pytestconfig):
-            plugin = pytestconfig.pluginmanager.get_plugin("pytest-kiwitcms")
-            assert plugin.api_url == "http://example.com"
-            assert plugin.username == "admin"
-            assert plugin.password == "secret"
-    """
-    )
-    result = testdir.runpytest("--kiwitcms", "-v")
-    result.stdout.fnmatch_lines(["*::test_file_configuration PASSED*"])
-    assert result.ret == 0
-
-
-def test_env_configuration(testdir, monkeypatch):
-    """If ~/.tcms.conf doesn't exist - use env variables."""
-    for envvar, value in DEFAULT_CONFIG:
-        monkeypatch.setenv(envvar, value)
-    testdir.makepyfile(
-        """
-        def test_env_configuration(pytestconfig):
-            plugin = pytestconfig.pluginmanager.get_plugin("pytest-kiwitcms")
-            assert plugin.api_url == "http://another.com"
-            assert plugin.username == "user"
-            assert plugin.password == "random"
-        """
-    )
-    result = testdir.runpytest("--kiwitcms", "-v")
-    result.stdout.fnmatch_lines(["*::test_env_configuration PASSED*"])
-    assert result.ret == 0
-
-
-@pytest.mark.parametrize("envvar, value", DEFAULT_CONFIG)
-@pytest.mark.usefixtures("config_file")
-def test_config_file_precedence(testdir, monkeypatch, envvar, value):
-    """Values from ~/.tcms.conf have higher priority than env variables."""
-    monkeypatch.setenv(envvar, value)
-    testdir.makepyfile(
-        """
-        def test_config_file_precedence(pytestconfig):
-            plugin = pytestconfig.pluginmanager.get_plugin("pytest-kiwitcms")
-            assert plugin.api_url == "http://example.com"
-            assert plugin.username == "admin"
-            assert plugin.password == "secret"
-        """
-    )
-    result = testdir.runpytest("--kiwitcms", "-v")
-    result.stdout.fnmatch_lines(["*::test_config_file_precedence PASSED*"])
-    assert result.ret == 0
-
-
-def test_empty_variable(testdir):
-    """Any empty value in config variables will lead to program exiting."""
-    testdir.makepyfile(
-        """
-        def test_empty_variable(pytestconfig):
-            plugin = pytestconfig.pluginmanager.get_plugin("pytest-kiwitcms")
-        """
-    )
-    result = testdir.runpytest("--kiwitcms", "-v")
-    result.stderr.fnmatch_lines(["Exit: Option api_url is empty"])
-    assert result.ret == 1
-
-
-def test_kiwitcms_pytest(testdir):
-    testdir.makepyfile(
-        """
-        import pytest
-        def test_pass():
-            assert 1 == 1
-        def test_fail():
-            assert 1 == 2
-        @pytest.mark.skip()
-        def test_skip():
-            assert 1 == 1
-        def test_error(test):
-            assert 1 == ""
-        """
-    )
-    testdir.runpytest("--kiwitcms")