MEDIUM: ssl: Set verify 'required' as global default for servers side.

If no CA file specified on a server line, the config parser will show an error.

Adds an cmdline option '-dV' to re-set verify 'none' as global default on
servers side (previous behavior).

Also adds 'ssl-server-verify' global statement to set global default to
'none' or 'required'.

WARNING: this changes the default verify mode from "none" to "required" on
the server side, and it *will* break insecure setups.
MAJOR: http: update connection mode configuration

At the very beginning of haproxy, there was "option httpclose" to make
haproxy add a "Connection: close" header in both directions to invite
both sides to agree on closing the connection. It did not work with some
rare products, so "option forceclose" was added to do the same and actively
close the connection. Then client-side keep-alive was supported, so option
http-server-close was introduced. Now we have keep-alive with a fourth
option, not to mention the implicit tunnel mode.

The connection configuration has become a total mess because all the
options above may be combined together, despite almost everyone thinking
they cancel each other, as judging from the common problem reports on the
mailing list. Unfortunately, re-reading the doc shows that it's not clear
at all that options may be combined, and the opposite seems more obvious
since they're compared. The most common issue is options being set in the
defaults section that are not negated in other sections, but are just
combined when the user expects them to be overloaded. The migration to
keep-alive by default will only make things worse.

So let's start to address the first problem. A transaction can only work in
5 modes today :
  - tunnel : haproxy doesn't bother with what follows the first req/resp
  - passive close : option http-close
  - forced close : option forceclose
  - server close : option http-server-close with keep-alive on the client side
  - keep-alive   : option http-keep-alive, end to end

All 16 combination for each section fall into one of these cases. Same for
the 256 combinations resulting from frontend+backend different modes.

With this patch, we're doing something slightly different, which will not
change anything for users with valid configs, and will only change the
behaviour for users with unsafe configs. The principle is that these options
may not combined anymore, and that the latest one always overrides all the
other ones, including those inherited from the defaults section. The "no
option xxx" statement is still supported to cancel one option and fall back
to the default one. It is mainly needed to ignore defaults sections (eg:
force the tunnel mode). The frontend+backend combinations have not changed.

So for examplen the following configuration used to put the connection
into forceclose :

    defaults http
        mode http
        option httpclose

    frontend foo.
        option http-server-close

  => http-server-close+httpclose = forceclose before this patch! Now
     the frontend's config replaces the defaults config and results in
     the more expected http-server-close.

All 25 combinations of the 5 modes in (frontend,backend) have been
successfully tested.

In order to prepare for upcoming changes, a new "option http-tunnel" was
added. It currently only voids all other options, and has the lowest
precedence when mixed with another option in another frontend/backend.
MEDIUM: http: make keep-alive + httpclose be passive mode

There's no particular reason for having keep-alive + httpclose combine
into forceclose when set in different frontend/backend sections, since
keep-alive does not close anything by default. Let's have this still
combination remain httpclose only.
MAJOR: http: switch to keep-alive mode by default

Since we support HTTP keep-alive, there is no more reason for staying
in tunnel mode by default. It is confusing for new users and creates
more issues than it solves. Option "http-tunnel" is available to force
to use it if really desired.

Switching to KA by default has implied to change the value of some
option flags and some transaction flags so that value zero (default)
matches keep-alive. That explains why more code has been changed than
expected. Tests have been run on the 25 combinations of frontend and
backend options, plus a few with option http-pretend-keepalive, and
no anomaly was found.

The relation between frontend and backends remains the same. Options
have been updated to take precedence over http-keep-alive which is now
implicit.

All references in the doc to haproxy not supporting keep-alive have
been fixed, and the doc for config options has been updated.
BUG/MEDIUM: http/auth: Sometimes the authentication credentials can be mix between two requests

The authentication function "get_http_auth()" extract credentials from
the request and keep it this values in shared cache. This function set
a flag in the session indicating that the authentication is already
parsed and the value stored in the cache are avalaible. If this flag is
set the authorization header is not re-parsed and the shared cache is
used.

If two request are simultaneous processsed, the first one check the
credentials. After this, the second request check also it's credentials
and change the data stored in the shared cache. When the first request
re-check credentials (for many reasons), they are changed. The change
can introduce a segfault.

This patch deactivate the cache upon success. When we need
authentication information from one request, they are re-parsed and
re-decoded. However, a failure to retrieve credentials is still
cached to avoid useless lookups.

This fix needs to be backported to 1.4 as well.
MINOR: doc: Bad cli function name.

The documentation describe a "disable" function,  but "enable" is
writed.
BUG/MEDIUM: http: fix regression caused by recent switch to keep-alive by default

Yesterday's commit 70dffda ("MAJOR: http: switch to keep-alive mode by default")
broke HTTP/1.0 handling without keep-alive when keep-alive is enabled both in
the frontend and in the backend.

Before this patch, it used to work because tunnel mode was the default one,
so if no mode was present in the frontend and a mode was set in the backend,
the backend was the first one to parse the header. This is what the original
patch tried to do with keep-alive by default, causing the version and the
connection header to be ignored if both the frontend and the backend were
running in keep-alive mode.

The fix consists in always parsing the header in non-tunnel mode, and
processing the rest of the logic in at least once, and again if the
backend works in a different mode than the frontend.

This is 1.5-specific, no backport is needed.
BUG/MEDIUM: listener: improve detection of non-working accept4()

On ARM, glibc does not implement accept4() and simply returns ENOSYS
which was not caught as a reason to fall back to accept(), resulting
in a spinning process since poll() would call again.

Let's change the error detection mechanism to save the broken status
of the syscall into a local variable that is used to fall back to the
legacy accept().

In addition to this, since the code was becoming a bit messy, the
accept4() was removed, so now the fallback code and the legacy code
are the same. This will also increase bug report accuracy if needed.

This is 1.5-specific, no backport is needed.
BUILD: listener: add fcntl.h and unistd.h

Otherwise it fails to build on some platforms.
MINOR: ssl: handshake optim for long certificate chains.

Suggested on the mailing list by Ilya Grigorik and greatly inspired
from Nginx code: we try to dynamicaly rise the output buffer size from
4k to 16k during the handshake to reduce the number of round trips.
This is mostly beneficial when initcwnd==10.

Ilya's tests confirm the gain and show a handshake time divided by 3 :

before:
   http://www.webpagetest.org/result/140116_VW_3bd95a5cfb7e667498ef13b59639b9bf/2/details/
after:
   http://www.webpagetest.org/result/140201_2X_03511ec63344f442b81c24d2bf39f59d/3/details/
BUG/MINOR: raw_sock: correctly set the MSG_MORE flag

Due to a typo, the MSG_MORE flag used to replace MSG_NOSIGNAL and
MSG_DONTWAIT. Fortunately, sockets are always marked non-blocking,
so the loss of MSG_DONTWAIT is harmless, and the NOSIGNAL is covered
by the interception of the SIGPIPE. So no issue could have been
caused by this bug.
MEDIUM: tcp-check new feature: connect

A new tcp-check rule type: connect.
It allows HAProxy to test applications which stand on multiple ports or
multiple applications load-balanced through the same backend.
[RELEASE] Released version 1.5-dev22

Released version 1.5-dev22 with the following main changes :
    - MEDIUM: tcp-check new feature: connect
    - MEDIUM: ssl: Set verify 'required' as global default for servers side.
    - MINOR: ssl: handshake optim for long certificate chains.
    - BUG/MINOR: pattern: pattern comparison executed twice
    - BUG/MEDIUM: map: segmentation fault with the stats's socket command "set map ..."
    - BUG/MEDIUM: pattern: Segfault in binary parser
    - MINOR: pattern: move functions for grouping pat_match_* and pat_parse_* and add documentation.
    - MINOR: standard: The parse_binary() returns the length consumed and his documentation is updated
    - BUG/MINOR: payload: the patterns of the acl "req.ssl_ver" are no parsed with the good function.
    - BUG/MEDIUM: pattern: "pat_parse_dotted_ver()" set bad expect_type.
    - BUG/MINOR: sample: The c_str2int converter does not fail if the entry is not an integer
    - BUG/MEDIUM: http/auth: Sometimes the authentication credentials can be mix between two requests
    - MINOR: doc: Bad cli function name.
    - MINOR: http: smp_fetch_capture_header_* fetch captured headers
    - BUILD: last release inadvertently prepended a "+" in front of the date
    - BUG/MEDIUM: stream-int: fix the keep-alive idle connection handler
    - BUG/MEDIUM: backend: do not re-initialize the connection's context upon reuse
    - BUG: Revert "OPTIM/MEDIUM: epoll: fuse active events into polled ones during polling changes"
    - BUG/MINOR: checks: successful check completion must not re-enable MAINT servers
    - MINOR: http: try to stick to same server after status 401/407
    - BUG/MINOR: http: always disable compression on HTTP/1.0
    - OPTIM: poll: restore polling after a poll/stop/want sequence
    - OPTIM: http: don't stop polling for read on the client side after a request
    - BUG/MEDIUM: checks: unchecked servers could not be enabled anymore
    - BUG/MEDIUM: stats: the web interface must check the tracked servers before enabling
    - BUG/MINOR: channel: CHN_INFINITE_FORWARD must be unsigned
    - BUG/MINOR: stream-int: do not clear the owner upon unregister
    - MEDIUM: stats: add support for HTTP keep-alive on the stats page
    - BUG/MEDIUM: stats: fix HTTP/1.0 breakage introduced in previous patch
    - Revert "MEDIUM: stats: add support for HTTP keep-alive on the stats page"
    - MAJOR: channel: add a new flag CF_WAKE_WRITE to notify the task of writes
    - OPTIM: session: set the READ_DONTWAIT flag when connecting
    - BUG/MINOR: http: don't clear the SI_FL_DONT_WAKE flag between requests
    - MINOR: session: factor out the connect time measurement
    - MEDIUM: session: prepare to support earlier transitions to the established state
    - MEDIUM: stream-int: make si_connect() return an established state when possible
    - MINOR: checks: use an inline function for health_adjust()
    - OPTIM: session: put unlikely() around the freewheeling code
    - MEDIUM: config: report a warning when multiple servers have the same name
    - BUG: Revert "OPTIM: poll: restore polling after a poll/stop/want sequence"
    - BUILD/MINOR: listener: remove a glibc warning on accept4()
    - BUG/MAJOR: connection: fix mismatch between rcv_buf's API and usage
    - BUILD: listener: fix recent accept4() again
    - BUG/MAJOR: ssl: fix breakage caused by recent fix abf08d9
    - BUG/MEDIUM: polling: ensure we update FD status when there's no more activity
    - MEDIUM: listener: fix polling management in the accept loop
    - MINOR: protocol: improve the proto->drain() API
    - MINOR: connection: add a new conn_drain() function
    - MEDIUM: tcp: report in tcp_drain() that lingering is already disabled on close
    - MEDIUM: connection: update callers of ctrl->drain() to use conn_drain()
    - MINOR: connection: add more error codes to report connection errors
    - MEDIUM: tcp: report connection error at the connection level
    - MEDIUM: checks: make use of chk_report_conn_err() for connection errors
    - BUG/MEDIUM: unique_id: HTTP request counter is not stable
    - DOC: fix misleading information about SIGQUIT
    - BUG/MAJOR: fix freezes during compression
    - BUG/MEDIUM: stream-interface: don't wake the task up before end of transfer
    - BUILD: fix VERDATE exclusion regex
    - CLEANUP: polling: rename "spec_e" to "state"
    - DOC: add a diagram showing polling state transitions
    - REORG: polling: rename "spec_e" to "state" and "spec_p" to "cache"
    - REORG: polling: rename "fd_spec" to "fd_cache"
    - REORG: polling: rename the cache allocation functions
    - REORG: polling: rename "fd_process_spec_events()" to "fd_process_cached_events()"
    - MAJOR: polling: rework the whole polling system
    - MAJOR: connection: remove the CO_FL_WAIT_{RD,WR} flags
    - MEDIUM: connection: remove conn_{data,sock}_poll_{recv,send}
    - MEDIUM: connection: add check for readiness in I/O handlers
    - MEDIUM: stream-interface: the polling flags must always be updated in chk_snd_conn
    - MINOR: stream-interface: no need to call fd_stop_both() on error
    - MEDIUM: connection: no need to recheck FD state
    - CLEANUP: connection: use conn_ctrl_ready() instead of checking the flag
    - CLEANUP: connection: use conn_xprt_ready() instead of checking the flag
    - CLEANUP: connection: fix comments in connection.h to reflect new behaviour.
    - OPTIM: raw-sock: don't speculate after a short read if polling is enabled
    - MEDIUM: polling: centralize polled events processing
    - MINOR: polling: create function fd_compute_new_polled_status()
    - MINOR: cli: add more information to the "show info" output
    - MEDIUM: listener: add support for limiting the session rate in addition to the connection rate
    - MEDIUM: listener: apply a limit on the session rate submitted to SSL
    - REORG: stats: move the stats socket states to dumpstats.c
    - MINOR: cli: add the new "show pools" command
    - BUG/MEDIUM: counters: flush content counters after each request
    - BUG/MEDIUM: counters: fix stick-table entry leak when using track-sc2 in connection
    - MINOR: tools: add very basic support for composite pointers
    - MEDIUM: counters: stop relying on session flags at all
    - BUG/MINOR: cli: fix missing break in command line parser
    - BUG/MINOR: config: correctly report when log-format headers require HTTP mode
    - MAJOR: http: update connection mode configuration
    - MEDIUM: http: make keep-alive + httpclose be passive mode
    - MAJOR: http: switch to keep-alive mode by default
    - BUG/MEDIUM: http: fix regression caused by recent switch to keep-alive by default
    - BUG/MEDIUM: listener: improve detection of non-working accept4()
    - BUILD: listener: add fcntl.h and unistd.h
    - BUG/MINOR: raw_sock: correctly set the MSG_MORE flag
DOC: add some information on capture.(req|res).hdr

Clarify the beginning of the index.
MINOR: http: capture.req.method and capture.req.uri

Add 2 sample fetchs allowing to extract the method and the uri of an
HTTP request.

FIXME: the sample fetches parser can't add the LW_REQ requirement, at
the moment this flag is used automatically when you use sample fetches.

Note: also fixed the alphabetical order of other capture.req.* keywords
in the doc.
BUG/MINOR: tcpcheck connect wrong behavior

A typo made first step of a tcpcheck to be a connect step. This patch
prevents this behavior. The bug was introduced in 1.5-dev22 with
"tcp-check connect" and only affects these directives. No backport is
needed.
MINOR: session: clean up the connection free code

Use conn_free() instead of pool_free2(conn...). This makes the code more
auditable.
MINOR: http: optimize capture.req.method and capture.req.uri

Useless strncpy were done in those two sample fetches, the
"struct chunk" allows us to dump the specified len.

The encode_string() in capture.req.uri was judged inappropriate and was
deleted.

The return type was fixed to SMP_T_CSTR.
BUG/MEDIUM: checks: immediately report a connection success

When no check type is configured (so the basic connection check), we
want the connection success to be immediately reported. Unfortunately,
it did not happen because in this case the connection is not registered
for read nor for write, and the wake_srv() callback does not handle this
case where no data transfer was requested. However, having option tcp-check
hides this problem because the check type follows a different setup mode,
by having check->type != 0 and the connection believing it must try to
send data.

The effect was that without any option, checks would succeed only at the
end of the check interval. So let's just add the wake-up condition.

This bug appeared with the recent polling changes, no backport is needed.
As a workaround, using "option tcp-check" fixes the problem.
MEDIUM: connection: don't use real send() flags in snd_buf()

This prevents us from passing other useful info and requires the
upper levels to know these flags. Let's use a new flags category
instead : CO_SFL_*. For now, only MSG_MORE has been remapped.