vo_x11: remove weird inactive code

I guess this code was supposed to handle cases like drawing RGBA as ARGB
by offsetting it by 1 byte.

The code didn't make any sense, though. It used to make sense before mpv
switched internal pixel formats from FourCCs to a simple enum. With the
FourCCs, "fmt | 128" selected the big endian version of a format. Of
course this doesn't work this way with the new pixel formats. It just so
happens that there are no formats with whose values match
IMGFMT_RGB32|128 or IMGFMT_BGR32|128, so this code was inactive.

All involved pixel formats seem to play fine on my setup (though it's
little endian only), and the code strictly matches the mpv pixel formats
against the format of the X image, so I'm not quite sure why this code
was there in the first place.

The original commit that added this was b333ae1 (svn 21602):
  Support for different endianness on client and server with -vo x11
vf_scale: uncrustify
vf_scale: don't byte-swap palette on big endian architectures

ffmpeg's and the internal palette format used to have different
endianess, but that is not the case anymore. This code was forgotten
when that change was made.
vf_scale: try to support all pixel formats

Until now, vf_scale only tried formats listed in the outfmt_list array.
Extend this and try every pixel format supported by mpv if trying
outfmt_list doesn't lead to success.

Also add some checks whether swscale really supports a given input or
output format. This was implicitly done with outfmt_list before.
vf_scale: remove rounding of sizes to 2 with 4:2:0

libswscale doesn't seem to require this (anymore?), and libavfiltert's
vf_scale doesn't do it either. Moreover, this wasn't done for most other
subsampled formats, not even very old ones. So just remove it.

(It'd be quite easy to align on chroma boundaries with all pixel
formats, though.)
sws_utils: refactor swscale wrapper code

This splits the monolithic mp_image_swscale() function into a bunch of
functions and a context struct. This means it's possible to set
arbitrary parameters (e.g. even obscure ones without getting in the
way), and you don't have to create the context on every call.

This code is preparation for removing duplicated libswscale API usage
from other parts of the code.
vf: make sure less important image params are set before/after filters

Image parameters like colorspace, color levels, and chroma location are
generally less important, and many filters don't set them correctly.
Force them instead in the generic VF code, which is probably better and
more convenient over all. So we designate this is a proper solution,
instead of a dirty hack.
vf_scale: use new swscale wrapper
mp_image: one utility function to set image parameters
vo_x11: use new swscale wrapper
sws_utils: remove some old code
sws_utils: don't recursively include libswscale header

Add libswscale includes where they are actually needed instead.
sws_utils: make hq swscale flags available with mp_sws_hq_flags

No need to duplicate this on the call-site.
img_format: add a mask for color class

Using the term "color class" to avoid confusion with the other
colorspace related concepts.

Also get rid of MP_IMGFLAG_FMT_MASK, since it was unused.
img_format: comment on some pixel formats
gl_video: add support for more rgb formats

Until now, only formats directly supported by OpenGL were supported.
This excludes various permutations of 8-bit RGB[A|0]. But we can simply
permutate the color channels in the shader, so do that. This also adds
support for all these weird RGB0 formats.

Note that we could use libavutil's pixfmt list instead of the
mp_packed_formats array, but trying to decrypt the pixfmt info would
probably end in pain, so this array with duplicated information is
actually better and shorter.

Note: I didn't actually test whether the alpha components are reproduced
correctly with alpha formats.
vo: remove aspdat.asp member

This was bad, because it was the only aspdat member updated by
vo_get_src_dst_rects() instead of vo_reconfig(). Now it isn't
accessed anymore, so remove it.
video: remove fullscreen flags chaos

There was a MPOpts fullscreen field, a mp_vo_opts.fs field, and
VOFLAG_FULLSCREEN. Remove all these and introduce a
mp_vo_opts.fullscreen flag instead.

When VOs receive VOCTRL_FULLSCREEN, they are supposed to set the
current fullscreen mode to the state in mp_vo_opts.fullscreen. They
also should do this implicitly on config().

VOs which are capable of doing so can update the mp_vo_opts.fullscreen
if the actual fullscreen mode changes (e.g. if the user uses the
window manager controls). If fullscreen mode switching fails, they
can also set mp_vo_opts.fullscreen to the actual state.

Note that the X11 backend does almost none of this, and it has a
private fs flag to store the fullscreen flag, instead of getting it
from the WM. (Possibly because it has to deal with broken WMs.)

The fullscreen option has to be checked on config() to deal with
the -fs option, especially with something like:

   mpv --fs file1.mkv --{ --no-fs file2.mkv --}

(It should start in fullscreen mode, but go to windowed mode when
playing file2.mkv.)

Wayland changes by: Alexander Preisinger <alexander.preisinger@gmail.com>
Cocoa changes by: Stefano Pigozzi <stefano.pigozzi@gmail.com>
wayland: don't resize in gl_wayland.c

The vo_wayland_fullscreen handles resizing for the video, because the video
could still be in fullscreen mode and resizing it in gl_wayland could make it
grow or shrink.