pytest-report-stream 0.1.0
A pytest plugin which allows to stream test reports at runtime
License: MIT
Requires Python: >=3.9

.. -*- mode: rst; coding: utf-8 -*-

============================================================
pytest-report-stream - Live stream test reports
============================================================

.. image:: https://img.shields.io/pypi/v/pytest-report-stream.svg
    :target: https://pypi.org/project/pytest-report-stream
    :alt: PyPI version

.. image:: https://img.shields.io/pypi/pyversions/pytest-report-stream.svg
    :target: https://pypi.org/project/pytest-report-stream
    :alt: Python versions


:Authors: Christos Liontos
:Version: 0.1.0
:Date:    2023-09-22
:Download: https://pypi.python.org/pypi/pytest-report-stream#downloads
:Code: https://github.com/kolitiri/pytest-report-stream


Welcome to pytest-report-stream!
============================================================
pytest-report-stream is a pytest plugin which allows to stream test reports at runtime.

It is a simple plugin the leverages pytest's build-in hook `pytest_runtest_makereport <https://docs.pytest.org/en/7.1.x/reference/reference.html#pytest.hookspec.pytest_runtest_makereport>`_ to intercept test execution and publish the status of the test run.

The plugin produces report events at specific moments:
- One report in the beginning of the test run
- One report in the end of each test
- One report in the end of the test run

The report structure looks like the dictionary below:

.. code-block:: python3

    {
        "test_run_tag": "My first test framework",
        "test_run_id": "5e080accaee748dc80619ee99245124e",
        "timestamps": {
            "started": datetime.datetime(2023, 10, 22, 17, 4, 43, 161646),
            "duration": 0.002042,
            "finished": datetime.datetime(2023, 10, 22, 17, 4, 43, 163688),
        },
        "summary": {"passed": 1, "failed": 1, "status": "in-progress"},
        "results": {
            "tests/test_module1": {
                "test_func_1": {
                    "name": "test_func_1",
                    "description": "Docstrings of function 1",
                    "status": "passed",
                    "error": "None",
                },
                "test_func_2": {
                    "name": "test_func_2",
                    "description": "Docstrings of function 2",
                    "status": "failed",
                    "error": "None",
                }
            }
        },
    }

Use cases
------------------------------------------------------------
The ``pytest-report-stream`` plugin can be particularly useful while running large and long lasting integration testing suites using pytest.

The plugin can be used to stream live report events to a remote service, allowing to monitor the progress and the status of the tests.

For example, Jenkins and other CI/CD tools are great, but there might be a requirement to aggregate test results from multiple builds.


Installation
============================================================
Install the plugin as below.

.. code-block:: sh

    pip install pytest-report-stream


Usage
============================================================
The plugin is available after installation and can be enabled using the ``--stream-reports`` flag.

.. code-block:: sh

    pytest --stream-reports

The plugin is using a ``report_client: ReportClient`` instance to generate the reports.

A default report_client is used to log the reports in STDOUT. (Note that in order to view the reports in STDOUT you will need to run ``pytest`` with the flag ``-s``)

The default client can be overriden in your own ``pytest_configure`` with an implementation of the abstract ``pytest_report_stream::ReportClient`` class.

Synchronous test using the default STDOUT report client
------------------------------------------------------------

.. code-block:: python3

    # content of tests/test_my_module.py
    def test_sync():
        pass

Aynchronous test using the default STDOUT report client
------------------------------------------------------------

.. code-block:: python3

    # content of tests/test_my_module.py
    import pytest

    @pytest.mark.asyncio
    async def test_async():
        pass

Aynchronous test using a custom report client
------------------------------------------------------------

.. code-block:: python3

    # content of tests/conftest.py
    import pytest
    from pytest_report_stream import ReportClient, ReportStreamPlugin


    class myCustomReportClient(ReportClient):
        async def publish_report(self, report_msg: dict) -> None:
            print('Some log comming from my custom report client')


    def pytest_configure(config):
        if config.option.stream_reports:
            config._stream_reports = ReportStreamPlugin(
                report_client=myCustomReportClient()
            )
            config.pluginmanager.register(config._stream_reports)

.. code-block:: python3

    # content of tests/test_my_module.py
    import pytest

    @pytest.mark.asyncio
    async def test_async():
        pass

You can implement the ``publish_report`` function and do pretty much anything, such as publishing the events to a message broker.


Requirements
============================================================
* pytest>=7.0.0
* pytest-asyncio


Contributing
============================================================
Contributions are very welcome.

Tests can be run with `tox <https://tox.wiki/en>`_, please ensure
the coverage at least stays the same before you submit a pull request.

.. code-block:: sh

    tox


License
============================================================
Distributed under the terms of the MIT license, "pytest-report-stream" is free and open source software


Issues
============================================================
If you encounter any problems, please `file an issue <https://github.com/kolitiri/pytest-report-stream/issues>`_ along with a detailed description.