libX11: check size of GetReqExtra after XFlush

Two users of GetReqExtra pass arbitrarily sized allocations from the
caller (ModMap and Host). Adjust _XGetRequest() (called by the GetReqExtra
macro) to double-check the requested length and invalidate "req" when
this happens. Users of GetReqExtra passing lengths greater than the Xlib
buffer size (normally 16K) must check "req" and fail gracefully instead
of crashing.

Any callers of GetReqExtra that do not check "req" for NULL
will experience this change, in the pathological case, as a NULL
dereference instead of a buffer overflow. This is an improvement, but
the documentation for GetReqExtra has been updated to reflect the need
to check the value of "req" after the call.

Bug that manifested the problem:
https://bugs.launchpad.net/ubuntu/+source/x11-xserver-utils/+bug/792628

Signed-off-by: Kees Cook <kees@outflux.net>
Reviewed-by: Alan Coopersmith <alan.coopersmith@oracle.com>
Signed-off-by: Alan Coopersmith <alan.coopersmith@oracle.com>
libX11: check "req" when calling GetReqExtra

This fixes the two callers of GetReqExtra to check that "req" is non-NULL
to avoid crashing now that GetReqExtra does internal bounds-checking on
the resulting buffer sizes.

Additionally updates comment describing return values to use names
instead of only literal values.

Signed-off-by: Kees Cook <kees@outflux.net>
Reviewed-by: Alan Coopersmith <alan.coopersmith@oracle.com>
Signed-off-by: Alan Coopersmith <alan.coopersmith@oracle.com>