Avoid repacking unreachable garbage in DfsGarbageCollector

If a repository has significant amounts of unreachable garbage the
final phase to coalesce it can take longer than any other part of the
garbage collection phase. Provide a setting for applications to tweak
the threshold where coalescing ends and files just remain on disk.

Change-Id: I5f11a998a7185c75ece3271d8bc6181bb83f54c1
Cluster UNREACHABLE_GARBAGE packs at the end of the search list

Garbage is unlikely to be used by a reader. Ensure they always
cluster at the end of the search list, no matter what timestamp
was used on the pack files.

Change-Id: I3bed89e9569ee3363c36bb3f73fcd34057a3883f
Simplify UploadPack by parsing wants separately from haves

The DHT backend was very slow at parsing objects. To work around
that performance limitation I obfuscated UploadPack by folding both
the want and have sets together in a single parse queue. Since DHT
was removed the complexity is no longer constructive to JGit.

Doing this refactoring prepares the code for a slightly future
change where the have lines need to be handled specially from the
want lines. Splitting the parsing up into two phases makes such
a modification trivial.

Change-Id: If7aad533b82448bbb688278e21f709282e5ccf4b
Avoid looking at UNREACHABLE_GARBAGE for client have lines

Clients send a bunch of unknown objects to UploadPack on each round
of negotiation. Many of these are not known to the server, which
leads the implementation to be looking at indexes for garbage packs.

Disable examining the index of a garbage pack, allowing servers to
avoid reading them from disk during negotiation.

The effect of this change is the server will only ACK a have line
if the object was reachable during the last garbage collection,
or was recently added to the repository. For most repositories
there is no impact in this behavior change.

If a repository rewinds a branch, runs GC, and then resets the
branch back to where it was before, the now current tip is going to
be skipped by this change. A client that has the commit may wind up
getting a slightly larger data transfer from the server as an older
common ancestor will be chosen during negotiation. This is fixable
on the server side by running GC again to correct the layout of
objects in pack files.

Change-Id: Icd550359ef70fc7b701980f9b13d923fd13c744b
A folder does not constitute a dirty work tree

This fixes two cases:
- A folder without tracked content exist both in the workdir and merged
commit, as long as there names within that folder does not conflict.
- An empty folder structure exists with the same name as a file in the
merged commit.

Bug: 402834
Change-Id: I4c5b9f11313dd1665fcbdae2d0755fdb64deb3ef
Add a NameRevCommand for describing IDs in terms of refnames

The walk logic does not use RevWalk because it needs to walk all paths
to each of the requested commits, keeping track of each path along which
the commit was found in the RevCommit subclass. From these paths, a
single "best" path is chosen based on the total path length, with a
penalty applied for paths that traverse merges.

This functionality parallels "git name-rev".

Change-Id: I92bfb47dd16c898313d2ee525395609c3bf72ebe
Merge "Add a NameRevCommand for describing IDs in terms of refnames"
Merge changes Icd550359,If7aad533

* changes:
  Avoid looking at UNREACHABLE_GARBAGE for client have lines
  Simplify UploadPack by parsing wants separately from haves
Merge "Avoid repacking unreachable garbage in DfsGarbageCollector"
Merge "Cluster UNREACHABLE_GARBAGE packs at the end of the search list"
Allow adding single refs or all tags to NameRevCommand

Change-Id: I90e85bc835d11278631afd0e801425a292578bba
NameRevCommand: Use ~ notation for first parents of merges

Prefer ~(N+1) to ^1~N. Although both are correct, the former is
cleaner and matches "git name-rev".

Change-Id: I772001a219e5eb346f5552c92e6d98c70b2cfa98
Simplfy caching of DfsPackDescription from PackWriter.Statistics

Let the pack description copy the relevant stats values. This
moves it out of the garbage collector and compactor algorithms,
co-locating with something that might care.

Remove some unnecessary code from the DfsPackCompactor, the stats
tracks the same information and can supply it.

Change-Id: Id64ab38d507c0ed19ae0d106862d175b7364eba3
Remove objects before optimization from DfsGarbageCollector

Just counting objects is not sufficient. There are some race
conditions with receive packs and delta base completion that
may confuse such a simple algorithm.

Instead always do the larger set computations, and rely on the
PackWriter having no objects pending as the way to avoid creating
an empty pack file.

Change-Id: Ic81fefb158ed6ef8d6522062f2be0338a49f6bc4
Remove cached_packs support in favor of bitmaps

The bitmap code in PackWriter knows exactly when to use a pack as
a "cached pack". It enables cached pack usage only when the pack
has a bitmap and its entire closure of objects needs to be sent.
This is a much simpler code path to maintain, and JGit actually
has a way to write the necessary index.

Change-Id: I2645d482f8733fdf0c4120cc59ba9aa4d4ba6881
Add toString() for RepoStatistics

This is helpful for writing the repository statistics into a log file.

Change-Id: I0e8cd9ad05f123ab3851960890a50213f353a373
Signed-off-by: Edwin Kempin <edwin.kempin@sap.com>
Add toString() for PackConfig

This is helpful for writing the pack configuration into a log file.

Change-Id: I5e7f5ff7e01c9538ca12a1860844ba9b467bdf05
Signed-off-by: Edwin Kempin <edwin.kempin@sap.com>
Merge "Add toString() for PackConfig"
Merge "A folder does not constitute a dirty work tree"
NameRevCommand: Don't use merge cost for first parent

Treat first parent traversals as 1 and higher parents as MERGE_COST,
to match git name-rev. Allow overriding the merge cost during tests to
avoid creating 2^16 commits on the fly.

Change-Id: I0175e0c3ab1abe6722e4241abe2f106d1fe92a69