(PUP-744) Create an HTTP site value object

This commit creates a site value object containing a scheme, host, and
port. A site identifies an HTTP(S) connection. It does not currently
take into account proxy host, port, user, password, as these are global
puppet settings.

The Site class defines the `eql?` and `hash` methods making it suitable
for use as the key in a hash table.
(PUP-744) Add method for moving a site to a new location

This commit adds a Site#move_to method that creates a new site pointing
to a new location. Note that we don't preserve the path from the new
site, as sites are only concerned with the HTTP connection layer, not
the HTTP request path and query parameters.
(PUP-744) Add helper method for use_ssl?

This commit adds a site helper method for use_ssl?
(PUP-744) Create an HTTP session

An HTTP session binds a puppet HTTP connection and an expiration time.
(PUP-744) Refactor to use Net::HTTP factory

Previously, puppet connections were directly responsible for creating
Net::HTTP objects. This was okay because there was only one code path
where new Net::HTTP objects needed to be created.

However, in order to support different connection pool behaviors,
caching and non-caching of Net::HTTP objects, we want to have two pool
implementations, and each of them will need the ability to create
Net::HTTP objects.

This commit creates a factory object for creating Net::HTTP objects,
and has the puppet connection delegate to it.
(PUP-744) Add dummy http pool

This commit adds a DummyPool that doesn't perform any caching of network
connections. It also adds the pool to the context system, but does so
using a Proc to defer evaluation of the pool. This way we are not
loading networking code while bootstrapping puppet.
(PUP-744) Create a pool of puppet HTTP connections

This commit adds a pool for caching puppet network connections. The pool
maintains a hash indexed by site.  Each site refers to an array of
sessions, where each session refers to a puppet HTTP connection and its
timeout.

It is expected the consumers of the pool will follow the pattern:

    conn = pool.take_connection(site)
    # do something with conn
    pool.add_connection(site, conn)

Whenever a connection is (re-)added to the pool, the pool will assign a
expiration timeout. When a caller next requests a connection, the pool
will lazily close any expired connection(s) for that site.

The timeout exists, because if there is a long delay between the time
that an open connection is added to the pool and when it is taken, the
remote peer may close the connection, e.g. apache. As a result, we
only want to cache connections for a short time. Currently the timeout
defaults to 15 seconds.
(PUP-744) Ensure connections do not leak

Previously, the dummy and caching pools allowed the caller to take a
connection from the pool and retain ownership of the connection. This is
undesirable because if the caller encounters an exception while using
the connection, then they have to remember to close it, otherwise the
underlying socket will leak.

There is also the issue that a caller may forget to add it back to the
pool before the pool is closed, in which case the socket will again
leak.

This commit changes the api to a block form to ensure connections are
always returned to the pool or closed.

Note the `reuse` flag is used to ensure the connection is returned to
the pool even if the caller does a non-local return:

    with_connection(..) { |conn| return }
(PUP-744) Retrieve Net::HTTP objects from the current pool

Previously, the puppet connection owned a ruby http connection, and
could create it lazily in a number of different ways, e.g. `#address`.

If an HTTP request resulted in a redirect, the `@connection` instance
variable would be overwritten with a new ruby http object. This made
it difficult to reason about the lifecycle of ruby http objects.

The commit associates a site with each puppet connection. As a result
it is trivial to ask the puppet connection what its `#address` is and
do so without creating a ruby http object.

This commit also removes the `@connection` instance variable. Instead
connections are obtained from the pool for the duration of the request,
e.g. `Connection#get`.

The `Connection#connection` private method is retained for now, as a
number of tests rely on it, via send(:connection). Currently it
delegates to the factory to return a new connection, but will be deleted
in a future commit.

Finally, the redirect loop relied on a non-local jump. This exposed a
bug in the pool implementation, because the connection wasn't returned
to the pool. That bug was fixed in a preceding commit, but to ensure
these sorts of things don't happen in the future, this commit returns
the response outside of the `with_connection` block.
(PUP-744) Cache Net::HTTP connections during configurer run

This commit overrides the http_pool binding stored in the context system
with a caching pool implementation. The caching pool is only used for
the duration of the configurer run, which includes both agent and apply
applications.

The pool is closed in an ensure block to ensure any cached connections
are closed. Also the `run_internal` method is marked as private to make
it clear which version callers should use.
(PUP-744) Rename DummyPool to NoCachePool

DummyPool was a bad name, renaming to what it actually does.
(PUP-744) Move connection factory into the pools

Previously, the puppet connection class owned the Net::HTTP factory and
passed it to the pool when requesting a connection. However, the caching
pool only needs the factory when creating a new connection.

This commit makes the factory stateless and pushes it into the
respective pools. The SSL verifier is stateful (contains peer certs)
and remains in puppet's connection class.

When a pool needs to create a connection, it calls back into the puppet
connection object to initialize SSL on the Net::HTTP object. In the case
of the caching poool, it needs to do this before starting the
connection, which will perform the TCP connect and SSL handshake
operations.
(PUP-744) Remove unnecessary after each routine

The puppet connection class doesn't have an ssl_host instance variable.
(PUP-744) Fix order of logging output

Previously, the "Starting connection" message came before the "Creating
connection" message. This commit reorders the messages to appear in the
correct order.
(PUP-744) Remove monkey patching of Net::HTTP#ca_file=

Commit 553b2ad8a (circa 2007) monkey patched Net::HTTP to set
`Net::HTTP#ca_file=`. This method is needed to specify the file
containing the set of CA cert(s) that the SSL client will use to
authenticate SSL servers.

Note the `ca_file=` method is available in all versions of ruby 1.8.7
and up[1], so it is no longer necessary to monkey patch this.

[1] https://github.com/ruby/ruby/blob/v1_8_7/lib/net/https.rb#L145
(PUP-744) Ensure connections we create support at least HTTP 1.1

When starting a persistent HTTP connection, we do not expliclty specify
`Connection: Keep-Alive`, because that is the default in HTTP/1.1[1].

This commit adds a test to ensure we are using HTTP 1.1 or later.
Amazingly ruby has defaulted to 1.1 as far back as ruby 1.8.7[2].

[1] http://tools.ietf.org/html/rfc7230#appendix-A.1.2
[2] https://github.com/ruby/ruby/blob/v1_8_7/lib/net/http.rb#L282
(PUP-744) Allow keepalive timeout to be configurable

Previously the http keepalive timeout was hard coded to be 15 seconds.
This commit adds a new puppet setting, `http_keepalive_timeout`, which
defaults to 4 seconds. The reason for the smaller value, is because
Apache 2.2 and 2.4 have a corresponding server side keepalive timeout,
which defaults to 5 seconds.

If the client timeout were greater than or equal to the server timeout,
then the client may try to reuse a connection from the pool that the
server has almost certainly closed.
(maint) Add setting to write http debug out to stderr

This commit adds a puppet setting, `http_debug`, enabling HTTP request
and responses to be written to stderr. It is disabled by default, and
should remain so in production environments, because it can leak
sensitive information.

If enabled, it displays information like:

    opening connection to puppetmaster...
    opened
    <- "GET /production/node/node?...\r\n"
    -> "HTTP/1.1 200 OK \r\n"
    -> "X-Puppet-Version: 3.6.2\r\n"
    -> "Content-Type: text/pson\r\n"
    -> "Server: WEBrick/1.3.1 (Ruby/1.9.3/2014-02-24) OpenSSL/1.0.1e\r\n"
    -> "Date: Fri, 18 Jul 2014 07:37:41 GMT\r\n"
    -> "Content-Length: 4202\r\n"
    -> "Connection: Keep-Alive\r\n"
    -> "\r\n"
    reading 4202 bytes...
    -> ""
    -> <response_body>
    read 4202 bytes
    Conn keep-alive
(maint) Update comment about request_* methods

The puppet connection class has request_get, etc methods that accept a
block. I tried to deprecate them, to eliminate the possibility that a
caller could borrow a connection from the pool indefinitely.

However, I found that we have to pass a block to ruby's request method
in order to stream the response body. If you don't pass a block, ruby
will always read the entire response body into memory. If you later
try to stream the body:

    response.read_body { |chunk| yield uncompressor.uncompress(chunk) } }

It will fail with:

    Net::HTTPOK#read_body called twice

This commit updates the comment so it's clear why these other
request_methods exist.
(PUP-744) Remove private Connection#connection method

Previously, the `Connection#connection` method was used in
`connection_spec.rb` to validate the state of the Net::HTTP object
created by the puppet connection.

Since the puppet connection no longer owns an Net::HTTP object (it may
use newly created or cached instances), the `#connection` method
doesn't make sense, and since it was private, can be removed.

The tests that relied on the private method have been updated, and in
some cases moved to the factory_spec, e.g. verifying proxy settings.