don't double free pkt

libgit2sharp has a use case that exercises this code path, but I could not make it happen myself.

@nulltoken do you have any insight into how this got triggered in that test?  I tried setting fetch specs to none, then to all, but it did not happen for me...!