vo_opengl: always dither

Dithering was disabled if the input bit depth was not larger than the
output bit depth of the screen framebuffer. But since scaling, RGB
conversion, and other filters change the number of significant bits
anyway, dithering could still benefit image quality even in these
cases. Always do dithering, unless dithering is completely disabled.

The original intention of this mechanism was not to change the image
needlessly when playing video that matches the native bit depth of the
screen.
sws_utils: use libswscale for GBRP output if available

libswscale added support for this recently.
vf_scale: fix warning
windows support: support 64-bit MS Windows in EXTERN_PREFIX definition

The EXTERN_PREFIX definition changed in 94b7db2 needs a separate case
for _WIN64, as MinGW defines both that and _WIN32 but there is no prefix
unlike 32-bit case.

Patch by redxii on http://devel.mplayer2.org/ticket/226
x11: cleanup, refactor

Move things that are used by vo_xv only into vo_xv, same for vo_x11.

Rename some functions exported by x11_common, like vo_init to
vo_x11_common. Make functions not used outsode of x11_common.c private
to that file. Eliminate all global variables defined by x11_common
(except error handler and colormap stuff).

There shouldn't be any functional changes, and only code is moved
around. There are some minor simplifications in the X11 init code, as
we completely remove the ability to initialize X11 and X11+VO
separately (see commit b4d9647 "mplayer: do not create X11 state in player frontend"),
and the respective functions are conflated into vo_x11_init() and
vo_x11_uninit().
x11_common: do not set custom error handler when calling XSelectInput

Do this because we want to remove the global variables required with the
old code. In particular, there doesn't seem any way to set a secure user
data pointer with xlib.

XSelectInput() causes a BadAccess error when some of the requested event
flags are reserved (for exclusive flags like ButtonPress). The custom
error handler caught this and set a global variable, so that the code
could retry the XSelectInput() call without the conflicting flags. Use a
different approach that doesn't need a custom error handler. (Although
we still assume that the error handler doesn't terminate the program.)
x11: reduce VO mode switching code duplication

Some parts for initiating mode switches were duplicated in every VO
supporting X11 (except vo_opengl/gl_common, which didn't support mode
switching). Move this to x11_common.c.

Note that this might be slightly risky: is it really guaranteed that no
VO needed to do "special" setup that depends on X parameters changing
after a mode switch, such as bit depth, visuals etc.? From what I can
see, this shouldn't be the case (X probably can't even change depth on
the fly). Even if this should be a one-way road, VM switching is in
general very useless, and its implementation buggy, so it can just be
removed should unfixable problems arise.
x11: fix some obvious video mode switching bugs

The original video mode wasn't reliably restored, so just store the
mode separately.

For some reason, window decorations got into the picture, possibly due
to an incorrect initial window position or something like this.
Normally, the window is positioned and sized such that it covers the
screen entirely, even though the window still has decorations and is
not in fullscreen mode (fullscreen wouldn't be correct, because the
virtual desktop size is not screen size). Hack-fix by forcing window
decorations off when VM-switching.

All in all, VM switching is still buggy and useless.
x11: simplify handling of X Visuals and Colormaps in VOs

Don't force VOs to pick an arbitrary default Visual and Colormap. They
still can override them if needed. This simplifies the X11 VO interface.

Always create a Colormap for simplicity. Using CopyFromParent fails if
the selected visual is not the same of that of the parent window, which
happens for me with vo_opengl.

vo_vdpau and vo_xv explicitly set CWBorderPixel, do that in x11_common
instead (it was already done for native windows, but not for slave mode
windows).

What gl_common did was incorrect in theory: freeing a colormap while a
window uses it will change the colormap of the window to "None", and
the color mapping for such windows is "undefined".
vo_x11: fix redrawing on expose events

This part wasn't converted when changing to the new OSD redrawing way.