generalize Acquire::GzipIndex
Use /var/empty as the homedir for _apt

Thanks to Axel Beckert
Revert making pkgAcquire::Item::DescURI() "const"

Revert because its a API change and the gain does not justify the
extra work to make the required changes in the consumers of this
interface at this point.
Remove check for "Translation-" from pkgAcqIndex::Custom600Headers()

This unneeded because pkgAcqIndexTrans has its own Custom600Headers()
method.
rewrite compressed indexes test to check with all compressors

Git-Dch: Ignore
Disable Mth.DropPrivsOrDie() in copy.cc for now

Dch-Ignore: true
Merge remote-tracking branch 'donkult/feature/generalize-gzipindex' into debian/sid
deprecate Pkg->Name in favor of Grp->Name

They both store the same information, so this field just takes up space
in the Package struct for no good reason. We mark it "just" as deprecated
instead of instantly removing it though as it isn't misleading like
Section was and is potentially used in the wild more often.
search for pkg names in the cache case-sensitive

Package names have to be lowercase (debian-policy §5.6.1) and in as
lowlevel as these method are it would be quiet strange to treat an
invalid package "suddently" as a valid one which other tools might or
might not accept. If case-insensitivity is really needed the frontend
should ensure this rather than these methods waste cpu cycles by
default.

Git-Dch: Ignore
packages in the cache are sorted by name so noise-free

Commit aa0fe657e46b87cc692895a36df12e8b74bb27bb sorts the package names
in the hashtable. We make use of this already in these functions, but as
a minor sideeffect it also means that we don't have 'noise' anymore
between packages belonging to the same group. We therefore don't need to
check for a matching name in Grp.FindPkg anymore.

Git-Dch: Ignore
count strings more accurately for stats

So far, only the few strings stored in stringitems were counted, but
many more strings are directly inserted into the cache. We account for
this now by identifying all these different strings and measure their
length. We are still not at the correct size of the cache in 'stats'
this way, but we are now again a bit closer.

Git-Dch: Ignore
drop stored StringItems in favor of in-memory mappings

Strings like Section names or architectures are needed vary often.
Instead of writing them each time we need them, we deploy sharing for
these special strings. Until now, this was done with a linked list of
strings in which we would search, which was stored in the cache.
It turns out we can do this just as well in memory as well with a bunch
of std::map's.

In memory means here that it isn't available anymore if we have a partly
invalid cache, but that isn't much of a problem in practice as the
status file is compared to the other files we parse very small and includes
mostly duplicates, so the space we would gain by storing is more or less
equal to the size of the stored linked list…
de-duplicate version strings in the cache

Turns out that version numbers aren't as random as you might guess.
In my cache for example, I have:
  Total package names: 69513 (1390 k)
  Total package structures: 188259 (9036 k)
  Total distinct versions: 186345 (13.4 M)
  Total dependencies: 2052242 (57.5 M)
which amounts to 1035873 (10,1 M) strings.
Reusing version strings reduces this to 161465 (3.479 k).

This comes at a cost of course: Generation is slightly slower, but we
are still faster than what we started with and it makes room (also cache
size wise) for further changes.
fix progress output for (dist-)upgrade calculation

Previously, we had a start and a done of the calculation printed by
higher-level code, but this got intermixed by progress reporting from an
external solver or the output of autoremove code…

The higherlevel code is now only responsible for instantiating a
progress object of its choosing (if it wants progress after all) and the
rest will be handled by the upgrade code. Either it is used to show the
progress of the external solver or the internal solver will give some
hints about its overall progress. The later isn't really a proper
progress as it will jump forward after each substep, but that is at
least a bit better than before without any progress indication.

Fixes also the 'strange' non-display of this progress line in -q=1, while
all others are shown, which is reflected by all testcase changes.
mark pkg(All|Dist)Upgrade as deprecated

The comment above their definition marks them already as such, so this
is only a formalisation of the deprecation and fixes the occurances we
have in our own code together with removing a magic number.

Git-Dch: Ignore
add specialisations for std::vector

Git-Dch: Ignore
add APT::PackageUniverse as a pkgCache wrapper

Git-Dch: Ignore
rework cachesets API to allow future extension

The introduction of Fnmatch showed that each new selector would require
multiple new virtual methods in the CacheSetHelper to work correctly,
which isn't that great. We now flip to a single virtual method which
handles all cases separated by an enum – as new enum values can be added
without an ABI break.

Great care was taken to make old code work with the new way of organisation,
which means in return that you might be bombarded with deprecation
warnings now if you don't adapt, but code should still compile and work
as before as can be seen in apt itself with this commit.

Git-Dch: Ignore
adapt to the new CacheSetHelper API

Git-Dch: Ignore
move PCI::From* methods into CacheSetHelper class

The methods itself deal with the helper a lot, so it makes sense to move
them to the helper itself, which helps also if we want to override some
of these methods, the FromString mentioned in the bugreport being the
obvious example.

VCI is spared from this change for now as while it would fit with the
same reasoning it much heavier entangled with the previous
CacheSetHelper change, so moving it now would mean breaking the API.
The PCI change is worthwhile on its own though as it is used by VCI.

Closes: 686221