Repository: opsdroid/opsdroid

Issue:
Migrate tests from unittest to pytest
Our existing test suite has been written with the Python [`unittest`](https://docs.python.org/3/library/unittest.html) framework. However, as the test suite has grown and opsdroid has become more complex we are running into issues with the tests. Mainly around setting up and tearing down tests.

The @opsdroid/maintainers team have decided that we want to migrate all tests to be written with the [`pytest`](https://docs.pytest.org/en/latest/contents.html) framework instead so that we can make better use of fixtures. Fixtures are more reusable and portable and should help reduce complexity all over.

There's a lot to be done but it can be done piece by piece as `pytest` can run tests in either format. So if you wish to help in the effort you can start by searching the codebase for unittest suites. These are classes which are subclassed from `unittest.TestCase` or `asynctest.TestCase`, so searching all files for `unittest.TestCase` and `asynctest.TestCase` should be a good place to start.

For detailed information on running the test suite and contributing to opsdroid [see the docs](https://docs.opsdroid.dev/en/latest/contributing/index.html). But the quickest way to get started us with [`tox`](https://tox.readthedocs.io/en/latest/).

```bash
pip install -U tox  # You only need to install tox once

tox -e py36,lint  # Run the Python 3.6 tests (the lowest version we support) and the linter
```

Once you have found a test suite you wish to convert there are a few steps you need to follow to convert from unittest to pytest:
- Move tests from top level `tests` directory to a nested `tests` directory in opsdroid. Create one in an appropriate place if there isn't already one.
- Remove test from class, pytest tests are just regular functions.
- Change assertions to use regular `assert` or [pytest assertions](https://docs.pytest.org/en/latest/assert.html).
- Mark async tests. In unittest we write async tests by using the `asynctest.TestCase`, but in pytest we decorate our tests with `@pytest.mark.asyncio` instead.
- Move setup operations to fixtures. If a test class contains a `setUp` method anything created here should become a fixture. Check the existing fixtures in `conftest.py` before creating new ones.
- Add docstrings to tests and fixtures to explain what they do. We have been pretty rubbish with this up until now and there are many tests which are not obvious in what they are testing.

Here's an example:

```python
# Before (unittest)
import asynctest
import asynctest.mock as mock

from opsdroid.cli.start import configure_lang
from opsdroid.core import OpsDroid
from opsdroid.events import Message
from opsdroid.matchers import match_regex
from opsdroid import constraints


class TestConstraints(asynctest.TestCase):
    """Test the opsdroid constraint decorators."""

    async def setUp(self):
        configure_lang({})

    async def getMockSkill(self):
        async def mockedskill(opsdroid, config, message):
            pass

        mockedskill.config = {}
        return mockedskill

    async def test_constrain_rooms_constrains(self):
        with OpsDroid() as opsdroid:
            skill = await self.getMockSkill()
            skill = match_regex(r".*")(skill)
            skill = constraints.constrain_rooms(["#general"])(skill)
            opsdroid.skills.append(skill)

            tasks = await opsdroid.parse(
                Message(text="Hello", user="user", target="#random", connector=None)
            )
            self.assertEqual(len(tasks), 2)  # Just match_always and match_event
```

```python
# After (pytest)
import pytest

from opsdroid.cli.start import configure_lang
from opsdroid.core import OpsDroid
from opsdroid.events import Message
from opsdroid.matchers import match_regex
from opsdroid import constraints


configure_lang({})  # Required for our internationalization of error messages


@pytest.fixture
def opsdroid():
    """An instance of the OpsDroid class."""
    with OpsDroid() as opsdroid:
        yield opsdroid


@pytest.fixture
def mock_skill():
    """A skill which does nothing but follows the skill API."""

    async def mockedskill(opsdroid, config, message):
        pass

    mockedskill.config = {}
    return mockedskill


@pytest.mark.asyncio
async def test_constrain_rooms_constrains(opsdroid, mock_skill):
    """Test that with the room constraint a skill is not called."""
    skill = match_regex(r".*")(mock_skill)
    skill = constraints.constrain_rooms(["#general"])(skill)
    opsdroid.skills.append(skill)

    tasks = await opsdroid.parse(
        Message(text="Hello", user="user", target="#random", connector=None)
    )
    assert len(tasks) == 2  # Just match_always and match_event
```

Fixtures such as the `opsdroid` fixture will be extremely useful in many tests, so we will be creating some standard ones like this which will be available in all tests. If you write a fixture that you think could be useful in other places please don't hesitate to contribute it to the `conftest.py` file.

If you have any questions you can chat to us on [Matrix](https://riot.im/app/#/room/#opsdroid-general:matrix.org). We encourage you to get started and if you have issues or get stuck then open a [draft PR](https://github.blog/2019-02-14-introducing-draft-pull-requests/) with your changes and we can take a look.

---

When this issue is complete the documentation about the migration should be removed from the testing docs.


Discussion:
@jacobtomlinson Can I claim it?
@tarangchikhalia you are welcome to start converting some tests and raise a PR. As I explained in the issue there are many tests to convert. So it will take a while and I expect many people will work on it, nobody can specifically claim this issue.
Hi @tarangchikhalia , 
How is this going? Have you written started making some progress on this one?
I found this to be a good first issue for me to start contributing, and I do not want us to do the same thing twice. Please let me know of the modules that you have already worked on so that I can start with something else.

(If there is a different communication channel for questions like this, please direct me to the correct place).

Thanks.
@abhishekmadhu as the issue states please just search the codebase for old tests and convert them over. There are many to do and ideally I would like to see lots of small PRs rather than large ones converting many tests. That way folks don't have to worry about treading on each others toes.

> So if you wish to help in the effort you can start by searching the codebase for unittest suites. These are classes which are subclassed from `unittest.TestCase` or `asynctest.TestCase`, so searching all files for `unittest.TestCase` and `asynctest.TestCase` should be a good place to start.
Hi @jacobtomlinson, I am hoping to contribute on this project starting from migratings tests from unittest to pytest. I have tried my hands on simple database_test module and opened a pull request, however there is a codecoverage fail for that. Does it needs to be attending or I can move to other test modules? 
I'm happy to resolve the issue. Please move on to more tests. 
Hi, this is my first time contributing and I am eager to help, I tried to convert test_database_sqlite.py to a pytest in a draft request so any feedback on this will be greatly appreciated. Also I just wanted to mention that the "see the docs" link in the first comment leads to a page that does not exist.
Hello @rr3khan thank you for pointing that to us. The [contributing documentation](https://docs.opsdroid.dev/en/stable/contributing.html) is here.

I am off to work but I'll try to have a look at the PR when I come back 😄👍
Fixed the link above, thanks!

Resolving patch:
diff --git a/opsdroid/conftest.py b/opsdroid/conftest.py
--- a/opsdroid/conftest.py
+++ b/opsdroid/conftest.py
@@ -1,8 +1,29 @@
 """Pytest config for all opsdroid tests."""
+import pytest
+
+import asyncio
+
 from opsdroid.testing import opsdroid
+from opsdroid.connector import Connector
 
 from opsdroid.cli.start import configure_lang
 
 __all__ = ["opsdroid"]
 
 configure_lang({})
+
+
+@pytest.fixture(scope="session")
+def get_connector():
+    def _get_connector(config={}):
+        return Connector(config, opsdroid=opsdroid)
+
+    return _get_connector
+
+
+@pytest.yield_fixture
+def event_loop():
+    """Create an instance of the default event loop for each test case."""
+    loop = asyncio.get_event_loop_policy().new_event_loop()
+    yield loop
+    loop.close()


Test patch:
diff --git a/opsdroid/tests/test_connector.py b/opsdroid/tests/test_connector.py
new file mode 100644
--- /dev/null
+++ b/opsdroid/tests/test_connector.py
@@ -0,0 +1,114 @@
+import pytest
+
+import asynctest.mock as amock
+
+from opsdroid.connector import Connector, register_event
+from opsdroid.events import Message, Reaction
+
+
+class TestConnectorBaseClass:
+    """Test the opsdroid connector base class."""
+
+    def test_init(self, opsdroid, get_connector):
+        connector = get_connector(config={"example_item": "test"})
+        assert connector.default_target is None
+        assert connector.name == ""
+        assert connector.config["example_item"] == "test"
+
+    def test_property(self, get_connector):
+        connector = get_connector(config={"name": "shell"})
+        assert connector.configuration.get("name") == "shell"
+
+    def test_connect(self, event_loop, get_connector):
+        with pytest.raises(NotImplementedError):
+            event_loop.run_until_complete(get_connector().connect())
+
+    def test_listen(self, event_loop, get_connector):
+        with pytest.raises(NotImplementedError):
+            event_loop.run_until_complete(get_connector().listen())
+
+    def test_respond(self, event_loop, get_connector):
+        with pytest.raises(TypeError):
+            event_loop.run_until_complete(get_connector().send(Message("")))
+
+    def test_unsupported_event(self, event_loop, get_connector):
+        with pytest.raises(TypeError):
+            event_loop.run_until_complete(get_connector().send(Reaction("emoji")))
+
+    def test_incorrect_event(self):
+        class NotAnEvent:
+            pass
+
+        class MyConnector(Connector):
+            @register_event(NotAnEvent)
+            def send_my_event(self, event):
+                pass
+
+        with pytest.raises(TypeError):
+            MyConnector()
+
+    def test_event_subclasses(self):
+        class MyEvent(Message):
+            pass
+
+        class MyConnector(Connector):
+            @register_event(Message, include_subclasses=True)
+            def send_my_event(self, event):
+                pass
+
+        connector = MyConnector({})
+        assert MyEvent in connector.events
+
+
+class TestConnectorAsync:
+    """Test the async methods of the opsdroid connector base class."""
+
+    @pytest.mark.asyncio
+    async def test_disconnect(self, get_connector):
+        res = await get_connector().disconnect()
+        assert res is None
+
+    @pytest.mark.asyncio
+    async def test_send_incorrect_event(self):
+        connector = Connector({"name": "shell"})
+        with pytest.raises(TypeError):
+            await connector.send(object())
+
+    @pytest.mark.asyncio
+    async def test_dep_respond(self, recwarn):
+        connector = Connector({"name": "shell"})
+        with amock.patch("opsdroid.connector.Connector.send") as patched_send:
+            await connector.respond("hello", room="bob")
+
+            assert len(recwarn) >= 1
+            assert recwarn.pop(DeprecationWarning)
+
+            assert patched_send.call_count == 1
+
+    @pytest.mark.asyncio
+    async def test_dep_react(self, get_connector, recwarn):
+        connector = get_connector({"name": "shell"})
+
+        with amock.patch("opsdroid.events.Message.respond") as patched_respond:
+            await connector.react(Message("ori"), "hello")
+
+            assert len(recwarn) >= 1
+            assert recwarn.pop(DeprecationWarning)
+
+            assert patched_respond.call_count == 1
+
+    @pytest.mark.asyncio
+    async def test_deprecated_properties(self, get_connector, recwarn):
+        connector = get_connector({"name": "shell"})
+
+        connector.default_target = "spam"
+        assert connector.default_room == "spam"
+
+        assert len(recwarn) >= 1
+        assert recwarn.pop(DeprecationWarning)
+
+        connector.default_room = "eggs"
+        assert connector.default_target == "eggs"
+
+        assert len(recwarn) >= 1
+        assert recwarn.pop(DeprecationWarning)
diff --git a/tests/test_helper.py b/opsdroid/tests/test_helper.py
similarity index 51%
rename from tests/test_helper.py
rename to opsdroid/tests/test_helper.py
--- a/tests/test_helper.py
+++ b/opsdroid/tests/test_helper.py
@@ -1,10 +1,6 @@
 import os
-import asyncio
 import datetime
 import tempfile
-import unittest
-import unittest.mock as mock
-import pytest
 
 from opsdroid.helper import (
     del_rw,
@@ -20,40 +16,39 @@
 )
 
 
-class TestHelper(unittest.TestCase):
+class TestHelper:
     """Test the opsdroid helper classes."""
 
-    def test_del_rw(self):
-        with mock.patch("os.chmod") as mock_chmod, mock.patch(
-            "os.remove"
-        ) as mock_remove:
-            del_rw(None, None, None)
-            self.assertTrue(mock_chmod.called)
-            self.assertTrue(mock_remove.called)
+    def test_del_rw(self, mocker):
+        mock_chmod = mocker.patch("os.chmod")
+        mock_remove = mocker.patch("os.remove")
+        del_rw(None, None, None)
+        assert mock_chmod.called is True
+        assert mock_remove.called is True
 
     def test_file_is_ipython_notebook(self):
-        self.assertTrue(file_is_ipython_notebook("test.ipynb"))
-        self.assertFalse(file_is_ipython_notebook("test.py"))
+        assert file_is_ipython_notebook("test.ipynb") is True
+        assert file_is_ipython_notebook("test.py") is False
 
     def test_convert_ipynb_to_script(self):
         notebook_path = os.path.abspath("tests/mockmodules/skills/test_notebook.ipynb")
 
         with tempfile.NamedTemporaryFile(mode="w", delete=False) as output_file:
             convert_ipynb_to_script(notebook_path, output_file.name)
-            self.assertTrue(os.path.getsize(output_file.name) > 0)
+            assert os.path.getsize(output_file.name) > 0
 
     def test_extract_gist_id(self):
-        self.assertEqual(
+        assert (
             extract_gist_id(
                 "https://gist.github.com/jacobtomlinson/"
                 "c9852fa17d3463acc14dca1217d911f6"
-            ),
-            "c9852fa17d3463acc14dca1217d911f6",
+            )
+            == "c9852fa17d3463acc14dca1217d911f6"
         )
 
-        self.assertEqual(
-            extract_gist_id("c9852fa17d3463acc14dca1217d911f6"),
-            "c9852fa17d3463acc14dca1217d911f6",
+        assert (
+            extract_gist_id("c9852fa17d3463acc14dca1217d911f6")
+            == "c9852fa17d3463acc14dca1217d911f6"
         )
 
     def test_opsdroid(self):
@@ -66,8 +61,8 @@ def test_convert_dictionary(self):
             {"name": "mattermost", "api-token": "test"},
         ]
         updated_modules = convert_dictionary(modules)
-        self.assertEqual(updated_modules["telegram"].get("token"), "test")
-        self.assertIn("token", updated_modules["mattermost"])
+        assert updated_modules["telegram"].get("token") == "test"
+        assert "token" in updated_modules["mattermost"]
 
     def test_get_config_option(self):
         module_config = {"repo": "test"}
@@ -76,10 +71,10 @@ def test_get_config_option(self):
             ["repo", "path", "gist"],
             module_config,
             True,
-            f"opsdroid.connectors.websocket",
+            "opsdroid.connectors.websocket",
         )
 
-        self.assertEqual(result, (True, "repo", "test"))
+        assert result == (True, "repo", "test")
 
     def test_get_config_no_option(self):
         module_config = {
@@ -92,7 +87,7 @@ def test_get_config_no_option(self):
             ["repo", "path", "gist"], module_config, True, "module"
         )
 
-        self.assertEqual(result, ("module", "module", "module"))
+        assert result == ("module", "module", "module")
 
     def test_get_config_option_exception(self):
         module_config = None
@@ -100,19 +95,16 @@ def test_get_config_option_exception(self):
         result = get_config_option(
             ["repo", "path", "gist"], module_config, True, "module"
         )
-        self.assertEqual(result, ("module", "module", "module"))
+        assert result == ("module", "module", "module")
 
 
-class TestJSONEncoder(unittest.TestCase):
+class TestJSONEncoder:
     """A JSON Encoder test class.
 
     Test the custom json encoder class.
 
     """
 
-    def setUp(self):
-        self.loop = asyncio.new_event_loop()
-
     def test_datetime_to_dict(self):
         """Test default of json encoder class.
 
@@ -124,31 +116,25 @@ def test_datetime_to_dict(self):
         test_obj = datetime.datetime(2018, 10, 2, 0, 41, 17, 74644)
         encoder = JSONEncoder()
         obj = encoder.default(o=test_obj)
-        self.assertEqual(
-            {
-                "__class__": type_cls.__name__,
-                "year": 2018,
-                "month": 10,
-                "day": 2,
-                "hour": 0,
-                "minute": 41,
-                "second": 17,
-                "microsecond": 74644,
-            },
-            obj,
-        )
+        assert {
+            "__class__": type_cls.__name__,
+            "year": 2018,
+            "month": 10,
+            "day": 2,
+            "hour": 0,
+            "minute": 41,
+            "second": 17,
+            "microsecond": 74644,
+        } == obj
 
 
-class TestJSONDecoder(unittest.TestCase):
+class TestJSONDecoder:
     """A JSON Decoder test class.
 
     Test the custom json decoder class.
 
     """
 
-    def setUp(self):
-        self.loop = asyncio.new_event_loop()
-
     def test_dict_to_datetime(self):
         """Test call of json decoder class.
 
@@ -168,55 +154,53 @@ def test_dict_to_datetime(self):
         }
         decoder = JSONDecoder()
         obj = decoder(test_obj)
-        self.assertEqual(datetime.datetime(2018, 10, 2, 0, 41, 17, 74644), obj)
-
+        assert datetime.datetime(2018, 10, 2, 0, 41, 17, 74644) == obj
 
-def test_get_parser_config():
-    parsers = [
-        {
-            "name": "dialogflow",
-            "module": "",
-            "config": {
+    def test_get_parser_config(self):
+        parsers = [
+            {
                 "name": "dialogflow",
                 "module": "",
-                "project-id": "test-ddd33",
-                "type": "parsers",
-                "enabled": True,
-                "entrypoint": None,
-                "is_builtin": "",
-                "module_path": "opsdroid.parsers.dialogflow",
-                "install_path": "",
-                "branch": "master",
+                "config": {
+                    "name": "dialogflow",
+                    "module": "",
+                    "project-id": "test-ddd33",
+                    "type": "parsers",
+                    "enabled": True,
+                    "entrypoint": None,
+                    "is_builtin": "",
+                    "module_path": "opsdroid.parsers.dialogflow",
+                    "install_path": "",
+                    "branch": "master",
+                },
+                "intents": None,
             },
-            "intents": None,
-        },
-        {
-            "name": "regex",
-            "module": "",
-            "config": {
+            {
                 "name": "regex",
                 "module": "",
-                "type": "parsers",
-                "enabled": True,
-                "entrypoint": None,
-                "is_builtin": "",
-                "module_path": "opsdroid.parsers.regex",
-                "install_path": "",
-                "branch": "master",
+                "config": {
+                    "name": "regex",
+                    "module": "",
+                    "type": "parsers",
+                    "enabled": True,
+                    "entrypoint": None,
+                    "is_builtin": "",
+                    "module_path": "opsdroid.parsers.regex",
+                    "install_path": "",
+                    "branch": "master",
+                },
+                "intents": None,
             },
-            "intents": None,
-        },
-    ]
-
-    dialogflow_config = get_parser_config("dialogflow", parsers)
+        ]
 
-    assert dialogflow_config
-    assert dialogflow_config["name"] == "dialogflow"
+        dialogflow_config = get_parser_config("dialogflow", parsers)
 
+        assert dialogflow_config
+        assert dialogflow_config["name"] == "dialogflow"
 
-def test_get_parser_config_none():
-    parsers = {}
+    def test_get_parser_config_none(self):
+        parsers = {}
 
-    config = get_parser_config("dialogflow", parsers)
+        config = get_parser_config("dialogflow", parsers)
 
-    assert not config
+        assert not config
diff --git a/tests/test_connector.py b/tests/test_connector.py
deleted file mode 100644
--- a/tests/test_connector.py
+++ /dev/null
@@ -1,119 +0,0 @@
-import unittest
-import asyncio
-
-import asynctest
-import asynctest.mock as amock
-
-from opsdroid.core import OpsDroid
-from opsdroid.connector import Connector, register_event
-from opsdroid.events import Message, Reaction
-from opsdroid.cli.start import configure_lang
-
-
-class TestConnectorBaseClass(unittest.TestCase):
-    """Test the opsdroid connector base class."""
-
-    def setUp(self):
-        self.loop = asyncio.new_event_loop()
-        configure_lang({})
-
-    def test_init(self):
-        config = {"example_item": "test"}
-        connector = Connector(config, opsdroid=OpsDroid())
-        self.assertEqual(None, connector.default_target)
-        self.assertEqual("", connector.name)
-        self.assertEqual("test", connector.config["example_item"])
-
-    def test_property(self):
-        opsdroid = amock.CoroutineMock()
-        connector = Connector({"name": "shell"}, opsdroid=opsdroid)
-        self.assertEqual("shell", connector.configuration.get("name"))
-
-    def test_connect(self):
-        connector = Connector({}, opsdroid=OpsDroid())
-        with self.assertRaises(NotImplementedError):
-            self.loop.run_until_complete(connector.connect())
-
-    def test_listen(self):
-        connector = Connector({}, opsdroid=OpsDroid())
-        with self.assertRaises(NotImplementedError):
-            self.loop.run_until_complete(connector.listen())
-
-    def test_respond(self):
-        connector = Connector({}, opsdroid=OpsDroid())
-        with self.assertRaises(TypeError):
-            self.loop.run_until_complete(connector.send(Message("")))
-
-    def test_unsupported_event(self):
-        connector = Connector({}, opsdroid=OpsDroid())
-        with self.assertRaises(TypeError):
-            self.loop.run_until_complete(connector.send(Reaction("emoji")))
-
-    def test_incorrect_event(self):
-        class NotanEvent:
-            pass
-
-        class MyConnector(Connector):
-            @register_event(NotanEvent)
-            def send_my_event(self, event):
-                pass
-
-        with self.assertRaises(TypeError):
-            MyConnector()
-
-    def test_event_subclasses(self):
-        class MyEvent(Message):
-            pass
-
-        class MyConnector(Connector):
-            @register_event(Message, include_subclasses=True)
-            def send_my_event(self, event):
-                pass
-
-        c = MyConnector({})
-        assert MyEvent in c.events
-
-
-class TestConnectorAsync(asynctest.TestCase):
-    """Test the async methods of the opsdroid connector base class."""
-
-    async def setup(self):
-        configure_lang({})
-
-    async def test_disconnect(self):
-        connector = Connector({}, opsdroid=OpsDroid())
-        res = await connector.disconnect()
-        assert res is None
-
-    async def test_send_incorrect_event(self):
-        connector = Connector({"name": "shell"})
-        with self.assertRaises(TypeError):
-            await connector.send(object())
-
-    async def test_dep_respond(self):
-        connector = Connector({"name": "shell"})
-        with amock.patch("opsdroid.connector.Connector.send") as patched_send:
-            with self.assertWarns(DeprecationWarning):
-                await connector.respond("hello", room="bob")
-
-            patched_send.call_count == 1
-
-    async def test_dep_react(self):
-        connector = Connector({"name": "shell"})
-        with amock.patch("opsdroid.events.Message.respond") as patched_respond:
-            with self.assertWarns(DeprecationWarning):
-                await connector.react(Message("ori"), "hello")
-
-            patched_respond.call_count == 1
-
-    async def test_depreacted_properties(self):
-        connector = Connector({"name": "shell"})
-
-        connector.default_target = "spam"
-        with self.assertWarns(DeprecationWarning):
-            assert connector.default_room == "spam"
-
-        with self.assertWarns(DeprecationWarning):
-            connector.default_room = "eggs"
-
-        assert connector.default_target == "eggs"