Repository: ishepard/pydriller
Pull request #117 (2020-05-03 13:09:09)

Ignore changes in unsupported languages in DMM
Commits can contain modifications in languages not supported by Lizard.
Such changes should be ignored when computing the delta maintainability metrics.

We add a predicate to `Modification`s checking whether the programming language used
is supported (this is an extension of the public API), implemented by a call to Lizard.

If none of the modifications of a commit is written in a supported language,
the dmm value returned becomes None (this makes the return type `Optional[Float]`
instead of just `Float`).

Also included are some minor improvements to the documentation.


Patch:
diff --git a/docs/deltamaintainability.rst b/docs/deltamaintainability.rst
index a04073b..d664952 100644
--- a/docs/deltamaintainability.rst
+++ b/docs/deltamaintainability.rst
@@ -19,9 +19,9 @@ Definition
 
 In one sentence, the delta-maintainability metric is the proportion of *low-risk change* in a commit. The resulting value ranges from 0.0 (all changes are risky) to 1.0 (all changes are low risk). It rewards making methods better, and penalizes making things worse.
 
-The starting point for the DMM is a *risk profile* [Heitlager2007]_. Traditionally, risk profiles categorize methods (or, more generally, also referred to as *units*) into four bins: low, medium, high, and very high risk methods. The risk profile of a class then is a 4-tuple (*l, m, h, v*) representing the amount (lines) of code in each of the four  categories.
+The starting point for the DMM is a *risk profile* [Heitlager2007]_. Traditionally, risk profiles categorize methods (or, more generally, also referred to as *units*) into four bins: low, medium, high, and very high risk methods. The risk profile of a class then is a 4-tuple (*l, m, h, v*) representing the amount of code (number of lines) in each of the four  categories.
 
-For simplicity, in the context of the DMM, only two bins are used: low risk, and non-low (medium, high, or very high) risk. To transfer risk profiles from file (or system) level to commit level, we consider *delta risk profiles*. These are pairs (*dl, dh*), with *dl* being the increase (or decrease) of low risk code, and *dl* the increase (or decrease) in high risk code.
+For simplicity, in the context of the DMM, only two bins are used: low risk, and non-low (medium, high, or very high) risk. To transfer risk profiles from file (or system) level to commit level, we consider *delta risk profiles*. These are pairs (*dl, dh*), with *dl* being the increase (or decrease) of low risk code, and *dh* the increase (or decrease) in high risk code.
 
 The delta risk profile can then be used to determine good and bad change:
 
@@ -41,7 +41,7 @@ The DMM can be used on arbitrary properties that can be determined at method (un
 - Unit **complexity**: Method cyclomatic complexity; low risk threshold 5.
 - Unit **interfacing**: Method number of parameters: low risk threshold 2.
 
-The original DMM paper also used coupling and cloned code as properties, but these are not easily computed per commit with the Lizard infrastructure. The thresholds have been determined empirically following the procedure described in [Alves2010]_.
+The original DMM paper also used coupling and cloned code as properties, but these are not easily computed per commit with the Lizard infrastructure. The thresholds are language-independent by design, and have been determined empirically following the procedure described in [Alves2010]_, using industrial benchmark data collected by SIG [SIG2019]_.
 
 Example usage
 =============
@@ -59,17 +59,20 @@ Collecting DMM values from a git repository is  straightforward::
 			commit.dmm_unit_interfacing
 			))
 
+The resulting ``dmm`` values are proportions with values between 0.0 and 1.0.
+Files that are changed in a commit, but which are written in languages not supported  by PyDriller (Lizard) are ignored -- these are often configuration (``.xml``, ``.yaml``) or documentation (``.txt``, ``.md``) files.
+If none of the files changed in a commit are in languages supported by Pydriller, the ``dmm`` value is ``None``.
+
 
 Under the hood
 ==============
 
 The main public API consists of the three ``dmm_unit_size``, ``dmm_unit_complexity``, and ``dmm_unit_interfacing`` properties on the ``Commit`` class, as illustrated above.
-
 Under the hood, the DMM implementation can be easily configured or accessed:
 
-- The thresholds are set as separate constants in the ``Commit`` class;
-- The main methods implementing the DMM  are parameterized with a predicate characterizing low risk methods: This makes it easy to  expand the implementation with other properties;
-- There  are separate (protected) methods to compute risk profiles and delta-risk profiles and ``Commit`` and ``Modification`` level, which can be used to conduct root cause analysis for lowly rated commits.
+- The thresholds are set as separate constants in the ``Method`` class;
+- The main methods implementing the DMM  are parameterized with an enum characterizing the DMM property of interest.
+- There are separate (protected) methods to compute risk profiles and delta-risk profiles at ``Commit`` and ``Modification`` level, which can be used to collect more detailed information for selected (e.g., lowly rated) commits.
 
 
 Relation to SIG DMM
@@ -82,7 +85,7 @@ PyDriller's OS-DMM and SIG's DMM differ in the following ways:
 - OS-DMM relies on Lizard for simple line counting, which also counts white space. SIG's DMM on the other hand ignores blank lines.
 - OS-DMM uses the thresholds as empirically determined by SIG, based on SIG's measurement methodology [Alves2010]_. OS-DMM's Lizard-based metric values may be different, and hence may classify methods in different risk bins for methods close to the thresholds.
 
-Consequently, individual DMM values are likely to differ a few percentages between the SIG DMM and OS-DMM implementations. However, in terms of trends and statistical analysis, the outcomes will likely be very similar.
+Consequently, individual DMM values are likely to differ a few percentage points between the SIG DMM and OS-DMM implementations. However, in terms of trends and statistical analysis, the outcomes will likely be very similar.
 Therefore:
 
 - For research purposes, we recommend the fully open PyDriller implementation ensuring reproducible results.
@@ -91,8 +94,10 @@ Therefore:
 References
 ==========
 
-.. [DiBiase2019] Marco di Biase, Ayushi Rastogi, Magiel Bruntink, and Arie van Deursen. **The Delta Maintainability Model: measuring maintainability of fine-grained code changes**. IEEE/ACM International Conference on Technical Debt (TechDebt) at ICSE 2019, pp 113-122 (`preprint <https://pure.tudelft.nl/portal/en/publications/the-delta-maintainability-model-measuring-maintainability-of-finegrained-code-changes(6ff67dee-2781-47d7-916f-bd36c5b61beb).html>`_, `doi <https://doi.org/10.1109/TechDebt.2019.00030>`_)
+.. [DiBiase2019] Marco di Biase, Ayushi Rastogi, Magiel Bruntink, and Arie van Deursen. **The Delta Maintainability Model: measuring maintainability of fine-grained code changes**. IEEE/ACM International Conference on Technical Debt (TechDebt) at ICSE 2019, pp 113-122 (`preprint <https://pure.tudelft.nl/portal/en/publications/the-delta-maintainability-model-measuring-maintainability-of-finegrained-code-changes(6ff67dee-2781-47d7-916f-bd36c5b61beb).html>`_, `doi <https://doi.org/10.1109/TechDebt.2019.00030>`_).
+
+.. [Heitlager2007] Ilja Heitlager, Tobias Kuipers, and Joost Visser. **A Practical Model for Measuring Maintainability**. 6th International Conference on the Quality of Information and Communications Technology, QUATIC 2007, IEEE, pp 30-39 (`preprint <http://wiki.di.uminho.pt/twiki/pub/Personal/Joost/PublicationList/HeitlagerKuipersVisser-Quatic2007.pdf>`_, `doi <https://doi.org/10.1109/QUATIC.2007.8>`_)
 
-.. [Heitlager2007] Ilja Heitlager, Tobias Kuipers, Joost Visser. **A Practical Model for Measuring Maintainability**. 6th International Conference on the Quality of Information and Communications Technology, QUATIC 2007, IEEE, pp 30-39 (`preprint <http://wiki.di.uminho.pt/twiki/pub/Personal/Joost/PublicationList/HeitlagerKuipersVisser-Quatic2007.pdf>`_, `doi <https://doi.org/10.1109/QUATIC.2007.8>`_)
+.. [Alves2010] Tiaga Alves, Christiaan Ypma, and Joost Visser. **Deriving metric thresholds from benchmark data**. IEEE International Conference on Software Maintenance (ICSM), pages 1–10. IEEE, 2010 (`preprint <http://wiki.di.uminho.pt/twiki/pub/Personal/Tiago/Publications/icsm10rt-alves.pdf>`_, `doi <https://doi.org/10.1109/ICSM.2010.5609747>`_).
 
-.. [Alves2010]  Tiaga Alves, Christiaan Ypma, and Joost Visser. **Deriving metric thresholds from benchmark data**. IEEE International Conference on Software Maintenance (ICSM), pages 1–10. IEEE, 2010. (`preprint <http://wiki.di.uminho.pt/twiki/pub/Personal/Tiago/Publications/icsm10rt-alves.pdf>`_, `doi <https://doi.org/10.1109/ICSM.2010.5609747>`_)
\ No newline at end of file
+.. [SIG2019] Reinier Vis, Dennis Bijslma, and Haiyun Xu. SIG/TÜViT Evaluation Criteria Trusted Product  Maintainability:  Guidance for producers. Version 11.0. Software Improvement Group, 2019 (`online <https://www.softwareimprovementgroup.com/wp-content/uploads/2019/11/20190919-SIG-TUViT-Evaluation-Criteria-Trusted-Product-Maintainability-Guidance-for-producers.pdf>`_).
\ No newline at end of file
diff --git a/pydriller/domain/commit.py b/pydriller/domain/commit.py
index 157df2d..50e8f53 100644
--- a/pydriller/domain/commit.py
+++ b/pydriller/domain/commit.py
@@ -24,6 +24,7 @@ from pathlib import Path
 from typing import List, Set, Dict, Tuple, Optional
 
 import lizard
+import lizard_languages
 from git import Diff, Git, Commit as GitCommit, NULL_TREE
 
 from pydriller.domain.developer import Developer
@@ -222,6 +223,17 @@ class Modification:  # pylint: disable=R0902
 
         return path.name
 
+    @property
+    def language_supported(self) -> bool:
+        """
+        Return whether the language used in the modification can be analyzed by Pydriller.
+        Languages are derived from the file  extension.
+        Supported languages are those supported by Lizard.
+
+        :return: True iff language  of this Modification can be analyzed.
+        """
+        return lizard_languages.get_reader_for(self.filename) is not None
+
     @property
     def nloc(self) -> int:
         """
@@ -267,7 +279,7 @@ class Modification:  # pylint: disable=R0902
         :return: Dictionary
         """
         lines = self.diff.split('\n')
-        modified_lines = {'added': [], 'deleted': []} # type: Dict[str, List[Tuple[int, str]]] 
+        modified_lines = {'added': [], 'deleted': []} # type: Dict[str, List[Tuple[int, str]]]
 
         count_deletions = 0
         count_additions = 0
@@ -374,6 +386,7 @@ class Modification:  # pylint: disable=R0902
         :param dmm_prop: Property indicating the type of risk
         :return: total delta risk profile for this property.
         """
+        assert self.language_supported
         low_before, high_before = self._risk_profile(self.methods_before, dmm_prop)
         low_after, high_after = self._risk_profile(self.methods, dmm_prop)
         return low_after - low_before, high_after - high_before
@@ -645,7 +658,7 @@ class Commit:
         return branches
 
     @property
-    def dmm_unit_size(self) -> float:
+    def dmm_unit_size(self) -> Optional[float]:
         """
         Return the Delta Maintainability Model (DMM) metric value for the unit size property.
 
@@ -657,12 +670,13 @@ class Commit:
         It penalizes (value close to 0.0) working on methods that remain large
         or get larger.
 
-        :return: The DMM value (between 0.0 and 1.0) for method size in this commit.
+        :return: The DMM value (between 0.0 and 1.0) for method size in this commit,
+                 or None if none of the programming languages in the commit are supported.
         """
         return self._delta_maintainability(DMMProperty.UNIT_SIZE)
 
     @property
-    def dmm_unit_complexity(self) -> float:
+    def dmm_unit_complexity(self) -> Optional[float]:
         """
         Return the Delta Maintainability Model (DMM) metric value for the unit complexity property.
 
@@ -675,11 +689,12 @@ class Commit:
         or get more complex.
 
         :return: The DMM value (between 0.0 and 1.0) for method complexity in this commit.
+                 or None if none of the programming languages in the commit are supported.
         """
         return self._delta_maintainability(DMMProperty.UNIT_COMPLEXITY)
 
     @property
-    def dmm_unit_interfacing(self):
+    def dmm_unit_interfacing(self) -> Optional[float]:
         """
         Return the Delta Maintainability Model (DMM) metric value for the unit interfacing property.
 
@@ -692,10 +707,11 @@ class Commit:
         or are extended with too many parameters.
 
         :return: The dmm value (between 0.0 and 1.0) for method interfacing in this commit.
-        """
+                  or None if none of the programming languages in the commit are supported.
+       """
         return self._delta_maintainability(DMMProperty.UNIT_INTERFACING)
 
-    def _delta_maintainability(self, dmm_prop: DMMProperty) -> float:
+    def _delta_maintainability(self, dmm_prop: DMMProperty) -> Optional[float]:
         """
         Compute the Delta Maintainability Model (DMM) value for the given risk predicate.
         The DMM value is computed as the proportion of good change in the commit:
@@ -706,17 +722,20 @@ class Commit:
         :param dmm_prop: Property indicating the type of risk
         :return: dmm value (between 0.0 and 1.0) for the property represented in the property.
         """
-        (delta_low, delta_high) = self._delta_risk_profile(dmm_prop)
-        dmm = self._good_change_proportion(delta_low, delta_high)
-        if delta_low < 0 and delta_high == 0:
-            assert dmm == 0.0
-            # special case where removal of good (low risk) code is OK.
-            # re-adjust dmm accordingly:
-            dmm = 1.0
-        assert 0.0 <= dmm <= 1.0
-        return dmm
+        delta_profile = self._delta_risk_profile(dmm_prop)
+        if delta_profile:
+            (delta_low, delta_high) = delta_profile
+            dmm = self._good_change_proportion(delta_low, delta_high)
+            if delta_low < 0 and delta_high == 0:
+                assert dmm == 0.0
+                # special case where removal of good (low risk) code is OK.
+                # re-adjust dmm accordingly:
+                dmm = 1.0
+            assert 0.0 <= dmm <= 1.0
+            return dmm
+        return None
 
-    def _delta_risk_profile(self, dmm_prop: DMMProperty) -> Tuple[int, int]:
+    def _delta_risk_profile(self, dmm_prop: DMMProperty) -> Optional[Tuple[int, int]]:
         """
         Return the delta risk profile of this commit, which a pair (dv1, dv2), where
         dv1 is the total change in volume (lines of code) of low risk methods, and
@@ -725,10 +744,13 @@ class Commit:
         :param dmm_prop: Property indicating the type of risk
         :return: total delta risk profile for this commit.
         """
-        deltas = [mod._delta_risk_profile(dmm_prop) for mod in self.modifications]  #pylint: disable=W0212
-        delta_low = sum(d[0] for d in deltas)
-        delta_high = sum(d[1] for d in deltas)
-        return delta_low, delta_high
+        supported_modifications = [mod for mod in self.modifications if mod.language_supported]
+        if supported_modifications:
+            deltas = [mod._delta_risk_profile(dmm_prop) for mod in supported_modifications]  #pylint: disable=W0212
+            delta_low = sum(dlow for (dlow, dhigh) in deltas)
+            delta_high = sum(dhigh for (dlow, dhigh) in deltas)
+            return delta_low, delta_high
+        return None
 
     @staticmethod
     def _good_change_proportion(low_risk_delta: int, high_risk_delta: int) -> float:


Test patch:
diff --git a/tests/test_dmm.py b/tests/test_dmm.py
index 18ae797..4bb0767 100644
--- a/tests/test_dmm.py
+++ b/tests/test_dmm.py
@@ -116,6 +116,16 @@ def test_dmm_unit_interfacing(repo: GitRepository, msg: str, dmm: float):
     commit = commit_by_msg(repo, msg)
     assert commit.dmm_unit_interfacing == dmm
 
+def test_unsupported_language(repo: GitRepository):
+    # Add .md file that cannot be analyzed by Lizard
+    commit = commit_by_msg(repo, 'Offer README explaining the repo purpose')
+    assert  commit.dmm_unit_size is None
+
+def test_mixin_unsupported_language(repo: GitRepository):
+    # Add .txt file and update .java files
+    commit = commit_by_msg(repo, 'Release under Apache 2 license')
+    assert  commit.dmm_unit_size == 1.0
+
 def test_delta_profile_modification(repo: GitRepository):
     commit = commit_by_msg(repo, 'Increase unit size to risky')
     mod = commit.modifications[0]
@@ -131,6 +141,12 @@ def test_delta_profile_commit(repo: GitRepository):
 
     assert commit._delta_risk_profile(DMMProperty.UNIT_SIZE) == (3, 1)
 
+def test_supported_languages(repo: GitRepository):
+    # Add .md file that cannot be analyzed by Lizard
+    commit = commit_by_msg(repo, 'Offer README explaining the repo purpose')
+    mod = commit.modifications[0]
+    assert not mod.language_supported
+
 @pytest.mark.parametrize(
     'dlo,dhi,prop', [
     ( 0,  0, 1.0),