Use X11_CFLAGS during the build so we find the right xlib headers.

We don't need x11_libs, because we don't directly call any xlib
functions.  We're just passing xlib types around.