Repository: getappmap/appmap-python (language: python)

Issue:
Add unittest integration



Discussion / hints:

Fix patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 857e0a32..2a2412ab 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+### Added
+- [#5] *unittest* integration.
+
 ### ## [0.1.0.dev10] - 2021-02-24
 ### Fixed
 - Choose the output directory on startup, make sure it's an absolute path.
@@ -47,13 +50,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   works properly for flaskbb, and seems more correct in general.
 
   Also, add Event.__repr__.
-    
+
 - Simplify (and fix) logging configuration
-    
+
   The code that configured logging was overly complex, and also buggy.  These changes
   simplify it, keep duplicate messages from being emitted, and properly allow per-module
   configuration of log level.
-  
+
 - Don't use inspect.isclass
 
   When testing to see if an object is a class, use type() instead of inpect.isclass. See
diff --git a/README.md b/README.md
index 9742f05f..fbde5df3 100644
--- a/README.md
+++ b/README.md
@@ -9,16 +9,16 @@ of code and key data flows.
 
 There are several ways to record AppMaps of your Python program using the `appmap` package:
 
-* Run your tests (pytest, unittest[coming soon]) with the environment variable
+* Run your tests (pytest, unittest) with the environment variable
   `APPMAP=true`. An AppMap will be generated for each test case.
-    
+
 * Use the `appmap.record` [context manager](#context-manager) to control recording. The context manager takes
   an instance of an `appmap.Recording`, which can be used to generate the AppMap.
 
 * [coming soon] Run your application server with AppMap remote recording enabled, and use
   the [AppLand browser extension](https://github.com/applandinc/appland-browser-extension)
   to start, stop, and upload recordings.
-  
+
 Once you have made a recording, there are two ways to view automatically generated
 diagrams of the AppMaps.
 
@@ -26,7 +26,7 @@ The first option is to load the diagrams directly in your IDE, using the [AppMap
 for VSCode](https://marketplace.visualstudio.com/items?itemName=appland.appmap).
 
 The second option is to upload them to the [AppLand server](https://app.land) using the
-[AppLand CLI](https://github.com/applandinc/appland-cli/releases).  
+[AppLand CLI](https://github.com/applandinc/appland-cli/releases).
 
 ### Supported versions
 
@@ -69,16 +69,16 @@ configuration excludes `app.mod2.MyClass`
 
 * `APPMAP_CONFIG` specifies the configuration file to use. Defaults to `appmap.yml` in the
   current directory
-  
+
 * `APPMAP_LOG_LEVEL` specifies log level to use, from the set `CRITICAL`, `ERROR`,
   `WARNING`, `INFO`, `DEBUG`. Not case-sensitive, defaults to `WARNING`.
-  
+
 * `APPMAP_OUTPUT_DIR` specifies the root directory for writing AppMaps. Defaults to
   `tmp/appmap`.
-  
+
 ## Test Frameworks
-`appmap-python` supports recording [pytest](https://pytest.org) and `unittest`[coming soon]
-test cases.
+`appmap-python` supports recording [pytest](https://pytest.org) and
+[unittest](https://docs.python.org/3/library/unittest.html) test cases.
 
 ### pytest
 `appmap-python` is a `pytest` plugin. When it's installed in a project that uses
@@ -118,9 +118,11 @@ PASSED
 ====================================================================== 1 passed in 0.45s ======================================================================
 ```
 
-### unittest [coming soon]
+### unittest
 `import appmap.unittest`. Instruments subclasses of `unittest.TestCase` and records each
-`test_*` function in the subclasses.
+`test_*` function in the subclasses. You can also use `python -m appmap.unittest` exactly like
+`python -m unittest` and leave your code unmodified (just remember to set the `APPMAP=true`
+environment variable).
 
 ### Run your tests
 Once you've configured your tests to generate AppMaps, run the tests with the
@@ -297,7 +299,7 @@ supported versions of Python and django:
 
 Note that `tox` requires the correct version of Python to be installed before it can
 create a test environment. [pyenv](https://github.com/pyenv/pyenv) is an easy way to
-manage multiple versions of Python. 
+manage multiple versions of Python.
 
 ### Code Coverage
 [coverage](https://coverage.readthedocs.io/) for coverage:


Test patch:
diff --git a/appmap/_implementation/testing_framework.py b/appmap/_implementation/testing_framework.py
new file mode 100644
index 00000000..dbd08b8f
--- /dev/null
+++ b/appmap/_implementation/testing_framework.py
@@ -0,0 +1,120 @@
+import inflection
+import re
+
+from contextlib import contextmanager
+from pathlib import Path
+
+from appmap._implementation import configuration, env, generation, recording
+from appmap._implementation.utils import fqname
+
+
+try:
+    # Make sure we have hooks installed if we're testing a django app.
+    # pylint: disable=unused-import
+    import appmap.django  # noqa: F401
+except ImportError:
+    pass  # probably no django
+
+
+class FuncItem:
+    def __init__(self, cls, name, method_id=None):
+        self.cls = cls
+        self.name = name
+        self.method_id = method_id or name
+
+    @property
+    def class_name(self):
+        return self.cls.__name__
+
+    @property
+    def defined_class(self):
+        return fqname(self.cls)
+
+    def is_in_class(self):
+        return self.cls is not None
+    has_feature_group = is_in_class
+
+    @property
+    def feature_group(self):
+        test_name = self.class_name
+        if test_name.startswith('Test'):
+            test_name = test_name[4:]
+        return inflection.humanize(inflection.titleize(test_name))
+
+    @property
+    def feature(self):
+        return inflection.humanize(self.test_name)
+
+    @property
+    def scenario_name(self):
+        name = '%s%s' % (self.feature[0].lower(), self.feature[1:])
+        if self.has_feature_group():
+            name = ' '.join([self.feature_group, name])
+        return name
+
+    @property
+    def test_name(self):
+        ret = self.name
+        return re.sub('^test.?_', '', ret)
+
+    @property
+    def filename(self):
+        fname = self.name
+        if self.cls:
+            fname = '%s_%s' % (self.defined_class, fname)
+        fname = re.sub('[^a-zA-Z0-9-_]', '_', fname)
+        if fname.endswith('_'):
+            fname = fname[:-1]
+        return fname
+
+    @property
+    def metadata(self):
+        ret = {}
+        if self.is_in_class():
+            ret['feature_group'] = self.feature_group
+            ret['recording'] = {
+                'defined_class': self.defined_class,
+                'method_id': self.method_id
+            }
+        ret.update({
+            'name': self.scenario_name,
+            'feature': self.feature
+        })
+        return ret
+
+
+class session:
+    def __init__(self, name, version=None):
+        if not env.enabled():
+            return
+
+        self.appmap_path = Path(configuration.Config().output_dir) / name
+        self.appmap_path.mkdir(parents=True, exist_ok=True)
+
+        framework = {'name': name}
+        if version is not None:
+            framework['version'] = version
+
+        self.metadata = {
+            'app': configuration.Config().name,
+            'frameworks': [framework],
+            'recorder': {
+                'name': name
+            }
+        }
+
+    @contextmanager
+    def record(self, klass, method, **kwds):
+        if not env.enabled():
+            yield
+            return
+
+        rec = recording.Recording()
+        with rec:
+            yield
+
+        fi = FuncItem(klass, method, **kwds)
+        filename = fi.filename + '.appmap.json'
+        metadata = fi.metadata
+        metadata.update(self.metadata)
+        (self.appmap_path / filename).write_text(generation.dump(rec, metadata))
diff --git a/appmap/pytest.py b/appmap/pytest.py
index 06b924b5..45b522e1 100644
--- a/appmap/pytest.py
+++ b/appmap/pytest.py
@@ -1,138 +1,18 @@
-import logging
 import pytest
-import os
-import os.path
-import re
 
-import inflection
+import appmap.unittest
+from appmap._implementation import testing_framework
 
-from appmap._implementation import configuration, env, generation, recording
-from appmap._implementation.utils import fqname
 
-logger = logging.getLogger(__name__)
-
-try:
-    # Make sure we have hooks installed if we're testing a django app.
-    # pylint: disable=unused-import
-    import appmap.django  # noqa: F401
-except ImportError:
-    pass  # probably no django
-
-
-class FuncItem:
-    def __init__(self, pyfuncitem):
-        self.item = pyfuncitem
-
-    @property
-    def class_name(self):
-        return self.item.cls.__name__
-
-    @property
-    def defined_class(self):
-        return fqname(self.item.cls)
-
-    @property
-    def method_id(self):
-        return self.item.originalname
-
-    def is_in_class(self):
-        return self.item.cls is not None
-    has_feature_group = is_in_class
-
-    @property
-    def feature_group(self):
-        test_name = self.class_name
-        if test_name.startswith('Test'):
-            test_name = test_name[4:]
-        return inflection.humanize(inflection.titleize(test_name))
-
-    @property
-    def feature(self):
-        return inflection.humanize(self.test_name)
-
-    @property
-    def scenario_name(self):
-        name = '%s%s' %(self.feature[0].lower(), self.feature[1:])
-        if self.has_feature_group():
-            name = ' '.join([self.feature_group, name])
-        return name
-
-    @property
-    def test_name(self):
-        ret = self.item.name
-        return re.sub('^test.?_', '', ret)
-
-    @property
-    def filename(self):
-        fname = self.item.name
-        if self.item.cls:
-            fname = '%s_%s' % (self.defined_class, fname)
-        fname = re.sub('[^a-zA-Z0-9-_]', '_', fname)
-        if fname.endswith('_'):
-            fname = fname[:-1]
-        return fname
-
-    @property
-    def metadata(self):
-        ret = {}
-        if self.is_in_class():
-            ret['feature_group'] = self.feature_group
-            ret['recording'] = {
-                'defined_class': self.defined_class,
-                'method_id': self.method_id
-            }
-        ret.update({
-            'name': self.scenario_name,
-            'feature': self.feature
-        })
-        return ret
-
-
-@pytest.hookimpl(hookwrapper=True)
+@pytest.hookimpl
 def pytest_runtestloop(session):
-    if not env.enabled():
-        yield
-        return
-
-    session.appmap_path = os.path.join(
-        configuration.Config().output_dir, 'pytest'
-    )
-    logger.debug('pytest_runtestloop, creating %s', session.appmap_path)
-    os.makedirs(session.appmap_path, exist_ok=True)
-    session.appmap_metadata = {
-        'app': configuration.Config().name,
-        'frameworks': [{
-            'name': 'pytest',
-            'version': pytest.__version__
-
-        }],
-        'recorder': {
-            'name': 'pytest'
-        }
-    }
-    yield
+    # Use the same session for unittest in case there are any unittest.TestCases.
+    appmap.unittest.session = session.appmap = testing_framework.session(name='pytest', version=pytest.__version__)
 
 
+# Note: don't use pytest_runtest_call as this will catch unittest tests.
+# They are recorded separately by appmap.unittest instead.
 @pytest.hookimpl(hookwrapper=True)
-def pytest_runtest_call(item):
-    if not env.enabled():
-        yield
-        return
-
-    session = item.session
-    item = FuncItem(item)
-    metadata = dict(session.appmap_metadata)
-    metadata.update(item.metadata)
-    logger.debug('pytest_pyfunc_call, metadata %s', repr(metadata))
-    fname = item.filename + '.appmap.json'
-    path = os.path.join(session.appmap_path, fname)
-
-    def write_recording(r):
-        with open(path, 'w') as appmap_file:
-            appmap_file.write(generation.dump(r, metadata))
-        logger.info('wrote recording %s', path)
-
-    logger.info('starting recording %s', path)
-    r = recording.Recording(exit_hook=write_recording)
-    with r:
+def pytest_pyfunc_call(pyfuncitem):
+    with pyfuncitem.session.appmap.record(pyfuncitem.cls, pyfuncitem.name, method_id=pyfuncitem.originalname):
         yield
diff --git a/appmap/test/data/pytest/expected.unittest.appmap.json b/appmap/test/data/pytest/expected.unittest.appmap.json
new file mode 100644
index 00000000..1d20ca90
--- /dev/null
+++ b/appmap/test/data/pytest/expected.unittest.appmap.json
@@ -0,0 +1,125 @@
+{
+  "version": "1.4",
+  "metadata": {
+    "language": {
+      "name": "python"
+    },
+    "client": {
+      "name": "appmap",
+      "url": "https://github.com/applandinc/appmap-python"
+    },
+    "app": "Simple",
+    "frameworks": [
+      {
+        "name": "unittest"
+      }
+    ],
+    "recorder": {
+      "name": "unittest"
+    },
+    "name": "Unit test test hello unitworld",
+    "feature": "Hello unitworld",
+    "feature_group": "Unit test test",
+    "recording": {
+      "defined_class": "test_simple.UnitTestTest",
+      "method_id": "test_hello_unitworld"
+    }
+  },
+  "events": [
+    {
+      "defined_class": "simple.Simple",
+      "method_id": "hello_world",
+      "path": "simple.py",
+      "lineno": 8,
+      "static": false,
+      "receiver": {
+        "class": "simple.Simple",
+        "value": "<simple.Simple object at 0xabcdef>"
+      },
+      "parameters": [],
+      "id": 2,
+      "event": "call",
+      "thread_id": 1
+    },
+    {
+      "defined_class": "simple.Simple",
+      "method_id": "hello",
+      "path": "simple.py",
+      "lineno": 2,
+      "static": false,
+      "receiver": {
+        "class": "simple.Simple",
+        "value": "<simple.Simple object at 0xabcdef>"
+      },
+      "parameters": [],
+      "id": 3,
+      "event": "call",
+      "thread_id": 1
+    },
+    {
+      "parent_id": 3,
+      "id": 4,
+      "event": "return",
+      "thread_id": 1
+    },
+    {
+      "defined_class": "simple.Simple",
+      "method_id": "world",
+      "path": "simple.py",
+      "lineno": 5,
+      "static": false,
+      "receiver": {
+        "class": "simple.Simple",
+        "value": "<simple.Simple object at 0xabcdef>"
+      },
+      "parameters": [],
+      "id": 5,
+      "event": "call",
+      "thread_id": 1
+    },
+    {
+      "parent_id": 5,
+      "id": 6,
+      "event": "return",
+      "thread_id": 1
+    },
+    {
+      "parent_id": 2,
+      "id": 7,
+      "event": "return",
+      "thread_id": 1
+    }
+  ],
+  "classMap": [
+    {
+      "name": "simple",
+      "type": "package",
+      "children": [
+        {
+          "name": "Simple",
+          "type": "class",
+          "children": [
+            {
+              "name": "hello",
+              "type": "function",
+              "location": "simple.py:2",
+              "static": false
+            },
+            {
+              "name": "hello_world",
+              "type": "function",
+              "location": "simple.py:8",
+              "static": false
+            },
+            {
+              "name": "world",
+              "type": "function",
+              "location": "simple.py:5",
+              "static": false
+            }
+          ]
+        }
+      ]
+    }
+  ]
+}
diff --git a/appmap/test/data/pytest/test_simple.py b/appmap/test/data/pytest/test_simple.py
index 660f379f..6c219c1b 100644
--- a/appmap/test/data/pytest/test_simple.py
+++ b/appmap/test/data/pytest/test_simple.py
@@ -1,6 +1,14 @@
 import os
+import unittest
+
 
 def test_hello_world():
     import simple
     os.chdir('/tmp')
     assert simple.Simple().hello_world() == 'Hello world!'
+
+
+class UnitTestTest(unittest.TestCase):
+    def test_hello_unitworld(self):
+        import simple
+        self.assertEqual(simple.Simple().hello_world(), 'Hello world!')
diff --git a/appmap/test/test_pytest.py b/appmap/test/test_pytest.py
index a73f4c1c..a29a253c 100644
--- a/appmap/test/test_pytest.py
+++ b/appmap/test/test_pytest.py
@@ -1,5 +1,6 @@
 import os
 import os.path
+from pathlib import Path
 
 import json
 
@@ -26,3 +27,15 @@ def test_basic_integration(self, data_dir, testdir):
             expected_appmap = json.load(f)
 
         assert generated_appmap == expected_appmap
+
+    def test_unittest_cases(self, testdir):
+        testdir.copy_example('pytest')
+        testdir.monkeypatch.setenv('APPMAP', 'true')
+
+        result = testdir.runpytest('-svv')
+        result.assert_outcomes(passed=2)
+
+        # unittest cases ran by pytest should get recorded as pytest tests
+        path = Path(str(testdir.tmpdir))
+        assert len(list(path.glob('tmp/appmap/pytest/*'))) == 2
+        assert len(list(path.glob('tmp/appmap/unittest/*'))) == 0
diff --git a/appmap/test/test_unittest.py b/appmap/test/test_unittest.py
new file mode 100644
index 00000000..9d2a942e
--- /dev/null
+++ b/appmap/test/test_unittest.py
@@ -0,0 +1,21 @@
+from pathlib import Path
+import json
+import sys
+
+from .appmap_test_base import AppMapTestBase
+
+
+class TestUnitTest(AppMapTestBase):
+    def test_basic_integration(self, data_dir, testdir):
+        testdir.copy_example('pytest')
+        testdir.monkeypatch.setenv('APPMAP', 'true')
+
+        testdir.run(sys.executable, '-m', 'appmap.unittest')
+
+        appmap_json = Path(str(testdir.tmpdir)) / 'tmp/appmap/unittest/test_simple_UnitTestTest_test_hello_unitworld.appmap.json'
+        generated_appmap = self.normalize_appmap(appmap_json.read_text())
+
+        appmap_json = Path(data_dir) / 'pytest/expected.unittest.appmap.json'
+        expected_appmap = json.loads(appmap_json.read_text())
+
+        assert generated_appmap == expected_appmap
diff --git a/appmap/unittest.py b/appmap/unittest.py
index e69de29b..d7d6eec7 100644
--- a/appmap/unittest.py
+++ b/appmap/unittest.py
@@ -0,0 +1,21 @@
+import unittest
+
+from appmap._implementation import testing_framework
+
+
+original_run = unittest.TestCase.run
+session = testing_framework.session('unittest')
+
+
+def run(self, result=None):
+    # There doesn't seem to be a public way to get to the test method name.
+    # pylint: disable=protected-access
+    with session.record(self.__class__, self._testMethodName):
+        original_run(self, result)
+
+
+unittest.TestCase.run = run
+
+
+if __name__ == '__main__':
+    unittest.main(module=None)