I'm actually running into this problem when building freeglut itself:
freeglut_internal.h:112:39: fatal error: X11/extensions/XInput.h: No such file or directory