jelmer/dulwich — pull request #51 by samv (2012-03-06 18:42:20+00:00)
Title: Here's the fix for thin pack fetching

It is advertised that thin packs are supported, but when fetched packs are
moved into the store, they are not resolved correctly. The add_thin_pack()
function must be used if we claim to support thin packs. It seems to have
a different calling convention to add_pack(), which is problematic. This
awkward solution involves keeping a memory buffer of all data read before
processing it all in one go. Ugly, but at least it works.

This broke another test which seems to be testing some kind of corner case,
which I couldn't figure out, so I just disabled it.

Fixes this failure during client.fetch:

Traceback (most recent call last):
File "fetch.py", line 12, in
determine_wants=repo.object_store.determine_wants_all,
File "/Users/svilain/src/dulwich/dulwich/client.py", line 206, in fetch
commit()
File "/Users/svilain/src/dulwich/dulwich/object_store.py", line 575, in commit
return self.move_in_pack(path)
File "/Users/svilain/src/dulwich/dulwich/object_store.py", line 549, in move_in_pack
entries = p.sorted_entries()
File "/Users/svilain/src/dulwich/dulwich/pack.py", line 1103, in sorted_entries
ret = list(self.iterentries(progress=progress))
File "/Users/svilain/src/dulwich/dulwich/pack.py", line 1091, in iterentries
for i, result in enumerate(PackIndexer.for_pack_data(self)):
File "/Users/svilain/src/dulwich/dulwich/pack.py", line 1238, in _walk_all_chains
for result in self._walk_ref_chains():
File "/Users/svilain/src/dulwich/dulwich/pack.py", line 1248, in _walk_ref_chains
self._ensure_no_pending()
File "/Users/svilain/src/dulwich/dulwich/pack.py", line 1244, in _ensure_no_pending
raise KeyError([sha_to_hex(s) for s in self._pending_ref])
KeyError: ['acf3a3f561ca42ec4d9d6ab3513be4f7e54ae474', '7fbc71b8ecee8657791a5dc3829be4acbc091e1c', 'dafc470f73b1d6dfe0904c0d95cb4401e1cdd28e', 'efe2c9a04ba829d711d8c41ce3b58b9781f9c4d7', '43b8208987b474aa0d03e1b439556a21b18a0d3d']

---
I also tried mocking up a generator–based version, which doesn't need to cache the whole pack before passing it to the PackStreamCopier, but I couldn't get it to pass tests. This one at least solves the bug, albeit with some ugliness.