(PUP-744) Ensure connections do not leak

Previously, the dummy and caching pools allowed the caller to take a
connection from the pool and retain ownership of the connection. This is
undesirable because if the caller encounters an exception while using
the connection, then they have to remember to close it, otherwise the
underlying socket will leak.

There is also the issue that a caller may forget to add it back to the
pool before the pool is closed, in which case the socket will again
leak.

This commit changes the api to a block form to ensure connections are
either returned to the pool. The `reuse` flag is used to ensure the
connection is returned to the pool even if the caller does a non-local
return.
(PUP-744) Retrieve Net::HTTP objects from the current pool

Previously, the puppet connection would lazily create a Net::HTTP
object from its factory. If an HTTP request resulted in a redirect,
the `@connection` instance variable would be overwritten with a new
Net::HTTP object. This made it difficult to reason about the lifecycle
of Net::HTTP objects, especially as some of the puppet connection
methods delegated to the current Net::HTTP object, e.g. `#address`,
which may not have been created yet.

The commit associates a site with each puppet connection. The site is
immutable and identifies the connection. As a result, it is trivial to
ask the puppet connection what its `#address` is. Secondly, it
simplifies the Net::HTTP lifecycle, since we only create them as they
are needed.

This commit changes the puppet connection to obtain Net::HTTP objects
from the current pool, rather than calling the factory directory. The
`with_connection` method ensures persistent HTTP connections are
started before being yielded, and are added back to the pool when
complete.

Unfortunately, the `Puppet::Network::HTTP::Connection#connection`
method is public (see tests). If it is invoked, we obtain a new
Net::HTTP object from the factory, but do not start it. We may want to
issue a deprecation warning.

Finally, the redirect loop needed to be modified, because the return
statement causes a non-local jump, which would prevent the
`with_connection` method from returning the connection to the pool.
(PUP-744) Cache Net::HTTP connections during configurer run

This commit overrides the http_pool binding stored in the context system
with a caching pool implementation. The caching pool is only used for
the duration of the configurer run, which includes both agent and apply
applications.

The pool is closed in an ensure block to ensure any cached connections
are closed. Also the `run_internal` method is marked as private to make
it clear which version callers should use.
(PUP-744) Rename DummyPool to NoCachePool

DummyPool was a bad name, renaming to what it actually does.
(PUP-744) Move connection factory into the pools

Previously, the puppet connection class owned the Net::HTTP factory and
passed it to the pool when requesting a connection. However, the caching
pool only needs the factory when creating a new connection.

This commit makes the factory stateless and pushes it into the
respective pools. The SSL verifier is stateful (contains peer certs)
and remains in puppet's connection class.

When a pool needs to create a connection, it calls back into the puppet
connection object to initialize SSL on the Net::HTTP object. In the case
of the caching poool, it needs to do this before starting the
connection, which will perform TCP connect and SSL handshake.
(PUP-744) Remove unnecessary after each routine
(PUP-744) Fix order of logging output

Previously, the "Starting connection" message came before the "Creating
connection" message. This commit reorders the messages so they make
sense.
(PUP-744) Remove monkey patching of Net::HTTP

Commit 553b2ad8a (circa 2007) monkey patched Net::HTTP to set
`Net::HTTP#ca_file=`. This method is needed to specify the file
containing the set of CA cert(s) that the SSL client will authenticate
SSL servers against.

Note the `ca_file=` method is available in all versions of ruby 1.8.7
and up[1], so it is no longer necessary to monkey patch this.

[1] https://github.com/ruby/ruby/blob/v1_8_7/lib/net/https.rb#L145
(PUP-744) Ensure connections we create support at least HTTP 1.1

Adds a test to ensure we create connection that support HTTP 1.1 or
1.2.
(PUP-744) Allow keepalive timeout to be configurable

Previously the http keepalive timeout was hard coded to be 15 seconds.
This commit adds a new puppet setting, `http_keepalive_timeout`, which
defaults to 4 seconds. The reason for the smaller value, is because
Apache 2.2 and 2.4 have a corresponding server side keepalive timeout,
which defaults to 5 seconds.

If the client timeout were greater than or equal to the server timeout,
then the client may try to reuse a connection from the pool that the
server has already closed.
(maint) Enable http debug output

This commit adds a puppet setting, `http_debug`, enabling HTTP request
and responses to be written to stderr. It is disabled by default, and
should remain so in production environments, because it can leak
sensitive information.

If enabled, it displays information like:

    opening connection to puppetmaster...
    opened
    <- "GET /production/node/arcturus.luna.lan?transaction_uuid=ee7e258d-66d0-43f7-a96f-8fea7ea7d19c&fail_on_404=true HTTP/1.1\r\nAccept: pson, msgpack, yaml, b64_zlib_yaml, raw\r\nUser-Agent: Ruby\r\nHost: puppetmaster:8140\r\n\r\n"
    -> "HTTP/1.1 200 OK \r\n"
    -> "X-Puppet-Version: 3.6.2\r\n"
    -> "Content-Type: text/pson\r\n"
    -> "Server: WEBrick/1.3.1 (Ruby/1.9.3/2014-02-24) OpenSSL/1.0.1e\r\n"
    -> "Date: Fri, 18 Jul 2014 07:37:41 GMT\r\n"
    -> "Content-Length: 4202\r\n"
    -> "Connection: Keep-Alive\r\n"
    -> "\r\n"
    reading 4202 bytes...
    -> ""
    -> <response_body>
    read 4202 bytes
    Conn keep-alive
(PUP-744) Deprecate request_{get,post,head} methods

Previously the request_{get,post,head} methods allowed a block to be
specified. However, they do not use the retry logic that the other
connection methods use.

The only consumer of these methods were the file content, when streaming
the response back from the server. However, there is no real need to use
a block, as the file content can be streamed from the response object.

This commit adds deprecation warnings to the old style methods, e.g.
`Connection#request_get` and updates the file content to use
`Connection#get`. As a result of this change file content requests will
follow HTTP redirects.
(PUP-744) Remove private Connection#connection method

Previously, the `Connection#connection` method was used in
`connection_spec.rb` to validate the state of the Net::HTTP object
created by the puppet connection.

Since the puppet connection doesn't own a single `Net::HTTP` object (it may
use newly created or cached instances), the `#connection` method
doesn't make sense, and since it was private, can be removed.

The tests that relied on the private method have been updated, and in
some cases moved to the factory_spec, e.g. verifying proxy settings.
(PUP-744) Don't modify site information when following redirects

Previously, the connection would mutate its host, port, etc when
following redirects. This wasn't really an issue, because connections
were always single use.

Now that connections can be cached, we do not want to mutate the
original site information.
(PUP-744) Fix off-by-one redirect limit error

Previously, the `redirect_limit` specified the maximum number of HTTP
requests that the connection would make, not the maximum number of
redirects that it would follow. As a result, a limit of 0 would prevent
any HTTP requests from being made. This was introduced in 60f22eb1 when
the redirect logic was added.

This commit changes the limit so that 0 means don't follow redirects,
1 means follow one redirect, etc.
(maint) Clarify HttpPool docs

We do use persistent HTTP connections, sometimes.
(maint) Remove pending test for releasing expired connections

Previously, I was concerned that a caller could borrow a connection
from the pool, and perform a long operation before returning it to the
pool. That could make it trivial for the caller to add expired
connections back to the pool, and cause issues for the next HTTP
request.

However, that is not likely, because connections are only borrowed for
the duration of the `Connection#get`, `#post`, etc calls, and those
methods don't take a block. The connection methods that do take a
block, e.g. `#request_get` are deprecated, and are not used within
puppet.
(maint) remove unnecessary expectations

Remove expectations that were secondary to the actual test, also
clarify behavior that was being tested.