BUG/MEDIUM: http/session: disable client-side expiration only after body

For a very long time, back in the v1.3 days, we used to rely on a trick
to avoid expiring the client side while transferring a payload to the
server. The problem was that if a client was able to quickly fill the
buffers, and these buffers took some time to reach the server, the
client should not expire while not sending anything.

In order to cover this situation, the client-side timeout was disabled
once the connection to the server was OK, since it implied that we would
at least expire on the server if required.

But there is a drawback to this : if a client stops uploading data before
the end, its timeout is not enforced and we only expire on the server's
timeout, so the logs report a 504.

Since 1.4, we have message body analysers which ensure that we know whether
all the expected data was received or not (HTTP_MSG_DATA or HTTP_MSG_DONE).
So we can fix this problem by disabling the client-side or server-side
timeout at the end of the transfer for the respective side instead of
having it unconditionally in session.c during all the transfer.

With this, the logs now report the correct side for the timeout. Note that
this patch is not enough, because another issue remains : the HTTP body
forwarders do not abort upon timeout, they simply rely on the generic
handling from session.c. So for now, the session is still aborted when
reaching the server timeout, but the culprit is properly reported. A
subsequent patch will address this specific point.

This bug was tagged MEDIUM because of the changes performed. The issue
it fixes is minor however. After some cooling down, it may be backported
to 1.4.

It was reported by and discussed with Rachel Chavez and Patrick Hemmer
on the mailing list.
BUG/MEDIUM: http: correctly report request body timeouts

This is the continuation of previous patch "BUG/MEDIUM: http/session:
disable client-side expiration only after body".

This one takes care of properly reporting the client-side read timeout
when waiting for a body from the client. Since the timeout may happen
before or after the server starts to respond, we have to take care of
the situation in three different ways :
  - if the server does not read our data fast enough, we emit a 504
    if we're waiting for headers, or we simply break the connection
    if headers were already received. We report either sH or sD
    depending on whether we've seen headers or not.

  - if the server has not yet started to respond, but has read all of
    the client's data and we're still waiting for more data from the
    client, we can safely emit a 408 and abort the request ;

  - if the server has already started to respond (thus it's a transfer
    timeout during a bidirectional exchange), then we silently break
    the connection, and only the session flags will indicate in the
    logs that something went wrong with client or server side.

This bug is tagged MEDIUM because it touches very sensible areas, however
its impact is very low. It might be worth performing a careful backport
to 1.4 once it has been confirmed that everything is correct and that it
does not introduce any regression.
BUG/MEDIUM: http: disable server-side expiration until client has sent the body

It's the final part of the 2 previous patches. We prevent the server from
timing out if we still have some data to pass to it. That way, even if the
server runs with a short timeout and the client with a large one, the server
side timeout will only start to count once the client sends everything. This
ensures we don't report a 504 before the server gets the whole request.

It is not certain whether the 1.4 state machine is fully compatible with
this change. Since the purpose is only to ensure that we never report a
server error before a client error if some data are missing from the client
and when the server-side timeout is smaller than or equal to the client's,
it's probably not worth attempting the backport.
MINOR: http: generic pointer to rule argument

Add a void *data which can be used as a generic storage for rule
arguments.
MEDIUM: listener: make the accept function more robust against pauses

During some tests in multi-process mode under Linux, it appeared that
issuing "disable frontend foo" on the CLI to pause a listener would
make the shutdown(read) of certain processes disturb another process
listening on the same socket, resulting in a 100% CPU loop. What
happens is that accept() returns EAGAIN without accepting anything.
Fortunately, we see that epoll_wait() reports EPOLLIN+EPOLLRDHUP
(likely because the FD points to the same file in the kernel), so we
can use that to stop the other process from trying to accept connections
for a short time and try again later, hoping for the situation to change.
We must not disable the FD otherwise there's no way to re-enable it.

Additionally, during these tests, a loop was encountered on EINVAL which
was not caught. Now if we catch an EINVAL, we proceed the same way, in
case the socket is re-enabled later.
BUILD: syscalls: remove improper inline statement in front of syscalls

Trying to build with an old gcc and glibc revealed that we must not
state "inline" in our _syscall* definitions since it's already present
in the declaration making use of the _syscall* macros.
BUILD: ssl: SSL_CTX_set_msg_callback() needs openssl >= 0.9.7

1.5-dev24 introduced SSL_CTX_set_msg_callback(), which came with OpenSSL
0.9.7. A build attempt with an older one failed and we're still compatible
with 0.9.6 in 1.5.
BUG/MAJOR: ssl: Fallback to private session cache if current lock mode is not supported.

Process shared mutex seems not supported on some OSs (FreeBSD).

This patch checks errors on mutex lock init to fallback
on a private session cache (per process cache) in error cases.
MAJOR: ssl: Change default locks on ssl session cache.

Prevously pthread process shared lock were used by default,
if USE_SYSCALL_FUTEX is not specified.

This patch implements an OS independant kind of lock:
An active spinlock is usedf if USE_SYSCALL_FUTEX is not specified.

The old behavior is still available if USE_PTHREAD_PSHARED=1.
BUG/MAJOR: session: recover the correct connection pointer in half-initialized sessions

John-Paul Bader reported a nasty segv which happens after a few hours
when SSL is enabled under a high load. Fortunately he could catch a
stack trace, systematically looking like this one :

(gdb) bt full
        level = 6
        conn = (struct connection *) 0x0
        err_msg = <value optimized out>
        s = (struct session *) 0x80337f800
        conn = <value optimized out>
        flags = 41997063
        new_updt = <value optimized out>
        old_updt = 1
        e = <value optimized out>
        status = 0
        fd = 53999616
        nbfd = 279
        wait_time = <value optimized out>
        updt_idx = <value optimized out>
        en = <value optimized out>
        eo = <value optimized out>
        count = 78
        sr = <value optimized out>
        sw = <value optimized out>
        rn = <value optimized out>
        wn = <value optimized out>

The variable "flags" in conn_fd_handler() holds a copy of connection->flags
when entering the function. These flags indicate 41997063 = 0x0280d307 :
  - {SOCK,DATA,CURR}_RD_ENA=1       => it's a handshake, waiting for reading
  - {SOCK,DATA,CURR}_WR_ENA=0       => no need for writing
  - CTRL_READY=1                    => FD is still allocated
  - XPRT_READY=1                    => transport layer is initialized
  - ADDR_FROM_SET=1, ADDR_TO_SET=0  => clearly it's a frontend connection
  - INIT_DATA=1, WAKE_DATA=1        => processing a handshake (ssl I guess)
  - {DATA,SOCK}_{RD,WR}_SH=0        => no shutdown
  - ERROR=0, CONNECTED=0            => handshake not completed yet
  - WAIT_L4_CONN=0                  => normal
  - WAIT_L6_CONN=1                  => waiting for an L6 handshake to complete
  - SSL_WAIT_HS=1                   => the pending handshake is an SSL handshake

So this is a handshake is in progress. And the only way to reach line 88
is for the handshake to complete without error. So we know for sure that
ssl_sock_handshake() was called and completed the handshake then removed
the CO_FL_SSL_WAIT_HS flag from the connection. With these flags,
ssl_sock_handshake() does only call SSL_do_handshake() and retruns. So
that means that the problem is necessarily in data->init().

The fd is wrong as reported but is simply mis-decoded as it's the lower
half of the last function pointer.

What happens in practice is that there's an issue with the way we deal
with embryonic sessions during their conversion to regular sessions.
Since they have no stream interface at the beginning, the pointer to
the connection is temporarily stored into s->target. Then during their
conversion, the first stream interface is properly initialized and the
connection is attached to it, then s->target is set to NULL.

The problem is that if anything fails in session_complete(), the
session is left in this intermediate state where s->target is NULL,
and kill_mini_session() is called afterwards to perform the cleanup.
It needs the connection, that it finds in s->target which is NULL,
dereferences it and dies. The only reasons for dying here are a problem
on the TCP connection when doing the setsockopt(TCP_NODELAY) or a
memory allocation issue.

This patch implements a solution consisting in restoring s->target in
session_complete() on the error path. That way embryonic sessions that
were valid before calling it are still valid after.

The bug was introduced in 1.5-dev20 by commit f8a49ea ("MEDIUM: session:
attach incoming connection to target on embryonic sessions"). No backport
is needed.

Special thanks to John for his numerous tests and traces.
DOC: add some explanation on the shared cache build options in the readme.

These ones become tricky, so better document them clearly.
MEDIUM: connection: Implement and extented PROXY Protocol V2

This commit modifies the PROXY protocol V2 specification to support headers
longer than 255 bytes allowing for optional extensions.  It implements the
PROXY protocol V2 which is a binary representation of V1. This will make
parsing more efficient for clients who will know in advance exactly how
many bytes to read.  Also, it defines and implements some optional PROXY
protocol V2 extensions to send information about downstream SSL/TLS
connections.  Support for PROXY protocol V1 remains unchanged.
BUG/MINOR: chunk: Fix function chunk_strcmp and chunk_strcasecmp match a substring.

They could match different strings as equal if the chunk was shorter
than the string. Those functions are currently only used for SSL's
certificate DN entry extract.
MINOR: ssl: add global statement tune.ssl.force-private-cache.

Boolean: used to force a private ssl session cache for each process in
case of nbproc > 1.
MINOR: ssl: remove fallback to SSL session private cache if lock init fails.

Now, haproxy exit an error saying:
Unable to initialize the lock for the shared SSL session cache. You can retry using
the global statement 'tune.ssl.force-private-cache' but it could increase the CPU
usage due to renegotiation if nbproc > 1.
MEDIUM: proxy: only adjust the backend's bind-process when already set

By default, a proxy's bind_proc is zero, meaning "bind to all processes".
It's only when not zero that its process list is restricted. So we don't
want the frontends to enforce the value on the backends when the backends
are still set to zero.
MEDIUM: config: limit nbproc to the machine's word size

Some consistency checks cannot be performed between frontends, backends
and peers at the moment because there is no way to check for intersection
between processes bound to some processes when the number of processes is
higher than the number of bits in a word.

So first, let's limit the number of processes to the machine's word size.
This means nbproc will be limited to 32 on 32-bit machines and 64 on 64-bit
machines. This is far more than enough considering that configs rarely go
above 16 processes due to scalability and management issues, so 32 or 64
should be fine.

This way we'll ensure we can always build a mask of all the processes a
section is bound to.
MEDIUM: config: check the bind-process settings according to nbproc

When a bind-process setting is present in a frontend or backend, we
now verify that the specified process range at least shares one common
process with those defined globally by nbproc. Then if the value is
set, it is reduced to the one enforced by nbproc.

A warning is emitted if process count does not match, and the fix is
done the following way :
  - if a single process was specified in the range, it's remapped to
    process #1
  - if more than one process was specified, the binding is removed
    and all processes are usable.

Note that since backends may inherit their settings from frontends,
depending on the declaration order, they may or may not be reported
as warnings.
MEDIUM: listener: parse the new "process" bind keyword

This sets the bind_proc entry in the bind_conf config block. For now it's
still unused, but the doc was updated.
MEDIUM: listener: inherit the process mask from the proxy

When a process list is specified on either the proxy or the bind lines,
the latter is refined to the intersection of the two. A warning is emitted
if no intersection is found, and the situation is fixed by either falling
back to the first process of the proxy or to all processes.