Make Proxy-Auto-Detect check for each host

When doing Acquire::http{,s}::Proxy-Auto-Detect, run the auto-detect
command for each host instead of only once. This should make using
"proxy" from libproxy-tools feasible which can then be used for PAC
style or other proxy configurations.

Closes: #759264
Avoid yielding blank lines with APT::Cmd::use-format=true
Add testcase for apt list --all-versions

Dch-Ignore: true
* apt-pkg/deb/dpkgpm.cc:
    - update string matching for dpkg I/O errors. (LP: #1363257)
    - properly parse the dpkg status line so that package name is properly set
      and an apport report is created. Thanks to Anders Kaseorg for the patch.
      (LP: #1353171)
Use heap to allocate PatternMatch to avoid potential stack overflow

When apt-cache search with many args (> 130) is given the allocation
of PatternMatch on the stack may fail resulting in a segmentation
fault. By using the heap the max size is much bigger and we also
get a bad_alloc expection instead of a segfault (which we can catch
*if* this ever becomes a pratical problem). No test for the crash
as its not reproducable with the MALLOC_ settings in framework.

Closes: 759612
Make Packages & Sources generation optional, during Generate call

refactor a bit, extract code out of Generate() into
DoGenerate{PackagesAndSources,Contents}, add new
APT::FTPArchive::ContentsOnly option to allow skipping the generation
of Package/Source files (if they are generated e.g. by some db outside
of apt-ftparchives control)
Run autopkgtest tests with "env -i" to avoid pollution from the host env

Closes: #759655
test/integration/test-ubuntu-bug-346386-apt-get-update-paywall: use downloadfile()
Fix incorrect upgradable listing in "apt list" (thanks to Michael Musenbrock)

The "apt list" command was using only the pkgDepCache but not the
pkgPolicy to figure out if a package is upgradable. This lead to
incorrect display of upgradable package when the user used the
policy to pin-down packages. Thanks to Michael Musenbrock for the
initial patch.

Closes: #753297
Ensure we have a Policy in CacheFile.BuildDepCache()

This partly reverts d059cc2 and fixes bug #753297 in a more
general way by ensuring that CacheFile.BuildDepCache() builds
a pkgPolicy if there isn't one already.
Improve Debug::Acquire::http debug output

Prefix all answers with the URL that the answer is for. This
helps when debugging and pipeline is enabled.
support regular expressions in 'apt search'

apt-cache search supported this since ever and in the code for apt was a
fixme indicating this should be added here as well, so here we go.
skip version if we already have this package as search-result

Git-Dch: Ignore
use a format string in ListSingleVersion

The method already deals with a format string, but had an else path
doing a hardcoded format as well. This is changed now to use the same
code for both - the format in the second case is still fixed though.

Git-Dch: Ignore
implement --full in apt search
make GetLocalitySortedVersionSet more generic

No reason in and of by itself at the moment, but prepares for the goal
of having 'apt search' and 'apt-cache search' using the same code now
that they at least support the same stuff. The 'apt' code is just a
multitude slower at the moment…

Git-Dch: Ignore
do use an 'unknown' arch-specification in test

Using 'kfreebsd' here makes the test fail on a kfreebsd system
(obviously), so we just use something totally madeup in the hope that
this is less like to conflict in the future.

Git-Dch: Ignore
detect terminal output with 'test -t' in tests

Instead of trying to inspect /proc and the fds inside we use "test -t 1"
instead as this is available and working on kfreebsd as well – not that
something breaks if we wouldn't, but we like color.

Git-Dch: Ignore
strip everything spacey in APT::String::Strip

Git-Dch: Ignore
fix and non-silent fail dpkg-overwrite error test

Commit cbcdd3ee9d86379d1b3a44e41ae8b17dc23111d0 removes the space at the
end of the debfile name dpkg send to us and we previously had included
in the pmerror message we printed on the statusfd.

Git-Dch: Ignore