Repository: gitpython-developers/GitPython

Issue #906: Possible performance regression in GitPython 3.0.0
Created: 2019-08-12T19:57:18Z
I have observed that the test run time on the Zuul project, which makes heavy use of GitPython, has increased by a factor of 1.5 (30 minutes under 2.1.13 to 45 minutes under 3.0.0).

Duplicate issue #907: 3.0.0 regression
Created: 2019-08-12T23:02:14Z
bisected to 0b6b90f which caused our tests to start failing with 3.0.0 release: datalad/datalad#3598 .  Unfortunately I do not have yet a good concise example to reproduce.
here is our failing test traceback on travis:
# from https://travis-ci.org/datalad/datalad/jobs/570888123#L1125
======================================================================
ERROR: datalad.support.tests.test_annexrepo.test_AnnexRepo_flyweight_monitoring_inode
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/datalad/tests/utils.py", line 615, in newfunc
    return t(*(arg + (filename,)), **kw)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/datalad/tests/utils.py", line 615, in newfunc
    return t(*(arg + (filename,)), **kw)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/datalad/support/tests/test_annexrepo.py", line 2399, in test_AnnexRepo_flyweight_monitoring_inode
    check_repo_deals_with_inode_change(AnnexRepo, path, store)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/datalad/support/tests/utils.py", line 53, in check_repo_deals_with_inode_change
    hexsha = repo.repo.head.object.hexsha
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/git/refs/symbolic.py", line 193, in _get_object
    return Object.new_from_sha(self.repo, hex_to_bin(self.dereference_recursive(self.repo, self.path)))
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/git/objects/base.py", line 64, in new_from_sha
    oinfo = repo.odb.info(sha1)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/git/db.py", line 37, in info
    hexsha, typename, size = self._git.get_object_header(bin_to_hex(sha))
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/git/cmd.py", line 1077, in get_object_header
    return self.__get_object_header(cmd, ref)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/git/cmd.py", line 1066, in __get_object_header
    return self._parse_object_header(cmd.stdout.readline())
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/git/cmd.py", line 1030, in _parse_object_header
    raise ValueError("SHA %s could not be resolved, git returned: %r" % (tokens[0], header_line.strip()))
ValueError: SHA b'8097085c9fb022ad18cea2ff099fbcd13ae953be' could not be resolved, git returned: b'8097085c9fb022ad18cea2ff099fbcd13ae953be missing'

Closing comment (2019-08-14T10:02:37Z): Also related to #906, which is a performance regression caused by the same commit 0b6b90f. The latter was introduced to fix #719 , and has since been reverted.
It's probably worth creating an intermediate 3.0.1 release to fix the breakage, and hope for another attempt to fix #719.
Also CC @bdauvergne who authored the fix and who might know why setting _git to None was required, and possible alternatives.
Thank you all for your help!