options: support parsing values into substructs

Add an alternate mode for option parser objects (struct m_config)
which is not inherently tied to any particular instance of an option
value struct. Instead, this type or parsers can be used to initialize
defaults in or parse values into a struct given as a parameter. They
do not have the save slot functionality used for main player
configuration. The new functionality will be used to replace the
separate subopt_helper.c parsing code that is currently used to parse
per-object suboptions in VOs etc.

Previously, option default values were handled by initializing them in
external code before creating a parser. This initialization was done
with constants even for dynamically-allocated types like strings.
Because trying to free a pointer to a constant would cause a crash
when trying to replace the default with another value, parser
initialization code then replaced all the original defaults with
dynamically-allocated copies. This replace-with-copy behavior is no
longer supported for new-style options; instead the option definition
itself may contain a default value (new OPTDEF macros), and the new
function m_config_initialize() is used to set all options to their
default values. Convert the existing initialized dynamically allocated
options in main config (the string options --dumpfile, --term-osd-esc,
--input=conf) to use this. Other non-dynamic ones could be later
converted to use this style of initialization too.

There's currently no public call to free all dynamically allocated
options in a given option struct because I intend to use talloc
functionality for that (make them children of the struct and free with
it).
options: make option struct the talloc parent of options

Allocate dynamically-allocated option values as talloc children of the
option struct. This will allow implementing per-object (VO etc)
options so that simply freeing the object will free associated options
too.

This doesn't change quite every allocation in m_option.c, but the
exceptions are legacy types which will not matter for new per-object
options.
VO: implement shared option handling, use for vdpau

Add infrastructure that allows VOs to specify the suboptions they
take, and get the values directly parsed into their private struct.
The option functionality available with the new system is the same as
for top-level player options. Convert vo_vdpau to use the new system
instead of the old subopt_helper.
options: fix specifying string options without parameter

Specifying a string option with no parameter, as in "--dumpfile" with
no '=', erroneously set the corresponding variable to NULL. Fix this
to give an error about missing parameter instead.

Suboption parsing explicitly treated empty option values as if the
option had been specified with no value (no '='). Thus it was not
possible to specify empty strings as values. I think this behavior was
originally added only because of other limitations in the old
implementation. Remove it, so that suboptions now behave the same as
top-level ones in this regard.

Document the NULL-distinguishing property of bstrdup0() that the code
depends on, and also make bstrdup() behave consistently.
build: use python3 to generate some files previously in git

Some files used during build are generated with Python scripts in
TOOLS/. Before, the generated files were included in the git tree.
Start creating them at build time. This introduces a build-dependency
on python3.

The files in question are:
libvo/vdpau_template.c
libmpdemux/ebml_types.h
libmpdemux/ebml_defs.c
build, codec-cfg.c: simplify builtin codecs.conf handling

The player can read codec mapping (codecs.conf) from an external file
or use embedded defaults. Before, the defaults were stored in the
player binary in the form of final already-parsed data structures.
Simplify things by storing the text of the codecs.conf file instead,
and parse that at runtime the same way an external file would be
parsed.

To create the previous parsed form, the build system first compiled a
separate binary named "codec-cfg", which parsed etc/codecs.conf and
then wrote the results as a C data structure that could be compiled
into the program. The new simple conversion of codecs.conf into a C
string is handled by the new script TOOLS/file2string.py.

After removing the codec-cfg binary, HOST_CC is no longer used for
anything. Remove the --host-cc configure option and associated logic.

Also remove the codec2html and codec-cfg-test functionality. Building
those was already broken and nobody cared.

There was a broken 3-character-long "fourcc" entry in etc/codecs.conf.
This happened to be accepted before but triggered a parse error after
the changes. Remove the broken entry and make the parsing functions
explicitly test for this error.
vo_vdpau: fix possible crash after preemption

Preemption recovery code could change the vc->vdp pointer when
recreating the VDPAU device. However, some other code cached the value
of vc->vdp in local variables over calls to handle_preemption(), and
could then crash when using the stale value later. Make the device
creation code keep the same vc->vdp instead of freeing and
reallocating it, so that the old pointer value is never invalidated
now.
Makefile: delete .d files in "clean" target too

Previously, .d files were only deleted by "distclean" target. I see no
reason why they should be kept by "clean". Delete them in that target
too.
docs: delete outdated translated manpages/docs

Delete all manpages and XML documentation in languages other than
English. The XML documentation was badly out of date. The content of
translated manpages was somewhat out of date, and manpage formatting
will change to use reStructuredText instead of raw troff format. If
updated translations are created for some languages later, I think
it's better to maintain those outside the main repository.
core: fix attempt to get audio pts without audio

written_audio_pts() can be called even if no audio track is active (at
least through get_current_time() when there's no known video PTS).
This triggered a crash due to NULL dereference. Add a check to return
MP_NOPTS_VALUE if no audio track exists.

Also remove a questionable update_osd_msg() call from per-file
initialization code. The call was at a point where an audio track
might be selected but not properly initialized, possibly also causing
a crash if update_osd_msg() queries current position. I don't see any
reason why the call would have been needed; it should get called
anyway before OSD contents are actually used for the new file.
video, audio: use lavc decoders without codecs.conf entries

Add support for using libavcodec decoders that do not have entries in
codecs.conf. This is currently only used with demux_lavf, and the
codec selection is based on codec_id returned by libavformat. Also
modify codec-related terminal output somewhat to make it use
information from libavcodec and avoid excessively long default output.

The new any-lavc-codec support is implemented with codecs.conf entries
that invoke vd_ffmpeg/ad_ffmpeg without directly specifying any
libavcodec codec name. In this mode, the decoders now instead select
the libavcodec codec based on codec_id previously set by demux_lavf
(if any). These new "generic" codecs.conf entries specify "status
buggy", so that they're tried after any specific entries with
higher-priority status.

Add new directive "anyinput" to codecs.conf syntax. This means the
entry will always match regardless of fourcc. This is used for the
above new codecs.conf entries (so the driver always gets to decide
whether to accept the input, and will fail init() if it can't find a
suitable codec in libavcodec). Remove parsing support for the obsolete
codecs.conf directive "cpuflags". This directive has not had any
effect and has not been used in default codecs.conf since many years
ago.

Shorten codec-related terminal output. When using libavcodec decoders,
show the libavcodec long_name field rather than codecs.conf "info"
field as the name of the codec. Stop showing the codecs.conf entry
name and "vfm/afm" name by default, as these are rarely needed;
they're now in verbose output only. Show "VIDEO:" line at VO
initialization rather than at demuxer open. This didn't really belong
in demuxer code; the new location may show more accurate values (known
after decoder has been opened) and works right if video track is
changed after initial demuxer open.

The vd.c changes (primarily done for terminal output changes) remove
round-to-even behavior from code setting dimensions based on aspect
ratio. I hope nothing depended on this; at least the even values were
not consistently guaranteed anyway, as the rounding code did not run
if the video file did not specify a nonzero aspect value.
demux, vd_ffmpeg: fix demux keyframe flag, set AV_PKT_FLAG_KEY

There was some confusion about the "flags" field in demuxer packets.
Demuxers set it to either 1 or 0x10 to indicate a keyframe (and the
field was not used to indicate anything else). This didn't cause
visible problems because nothing read the value. Replace the "flags"
field with a boolean "keyframe" field. Set AV_PKT_FLAG_KEY based on
this field in packets fed to libavcodec video decoders (looks like PNG
and ZeroCodec are the only ones which depend on values from demuxer;
previously this was hardcoded to true for PNG).

Make demux_mf set the keyframe field in every packet. This matters for
PNG files now that the demuxer flag is forwarded to libavcodec.

Fix logic setting the field in demux_mkv. It had probably not been
updated when adding SimpleBlock support. This probably makes no
difference for any current practical use.
vd_vfw.c: fix part missing from previous commit

vd_vfw.c was missing a change to rename field used from 'flags' to
'keyframe'. This broke compilation with 32-bit Windows codec support
enabled.
demux_lavf: use lavf for RealMedia (.rm) files by default

RealMedia was listed as a format for which the internal demuxer
(demux_real) was preferred over lavf. The original reason for this
(lavf failing to give any timing information for some video frames)
has been fixed in libavformat since. Make demux_lavf the preferred
demuxer for RealMedia.

The libavformat demuxer does still have issues. COOK audio initially
misbehaves after a seek (inconsistent timestamps, audio remaining from
the before-seek position). However, the internal demuxer seemed to be
_consistently_ out of sync with a test file. I haven't done thorough
testing, but the internal demuxer does not seem less buggy.
options: support "no-foo" syntax for flag suboptions

Allow using "no-foo" as an alternative to "foo=no" for flag
suboptions, similarly to what top-level flag options already support.
This means things like "--lavdopts=no-fast" or
"--vo=vdpau:no-chroma-deint" are now supported.
vo_vdpau: disable refresh-aware frame timing when composited

Under a compositing window manager the current VDPAU implementation
behaves differently than without it. Frame flip timing info becomes
incorrect (I guess it only reflects when the frame was sent to the
compositor, not when it was actually shown), and there is no
limitation to at most one frame switch per refresh like without
compositing. Detect whether a compositing window manager is active and
disable refresh-aware frame timing and dropping in this case,
similarly to what fps=-1 would do. This behavior can be controlled
with the new suboption "composite-detect".

Disabling the refresh-aware logic makes timing somewhat less accurate.
Because the video switch rate limit isn't there, the lack of frame
dropping on player side does not impose a hard limit on video FPS, but
does reduce performance somewhat as redundant frames are drawn in
memory.

The existence of a compositing window manager does not guarantee that
the current window is actually composited, so the current check is not
foolproof. In particular, some WMs have support for a "unredirect
fullscreen windows" option. Support for such things could be improved.
demux_rawdv.c: fix change missing from recent 8079f4ff821

Like vd_vfw.c fixed earlier, demux_rawdv.c was also missing a change
needed for the "flags"->"keyframe" demux packet field rename. This
broke compilation with libdv enabled.

Hopefully there aren't more cases. It's hard to reliably check for
references in files that aren't compiled in the local tree if they use
different base variable names, as the "flags" name itself is used for
many unrelated things.
vo_corevideo, vo_sharedbuffer: put private state in vo->priv

These VOs were already using a struct for all private data but the
struct variable itself was static. Change them to store the address in
vo->priv. Also change them to use the new automatic private data
allocation and option parsing mechanism.
vo_sharedbuffer: remove from autoprobe list

This video output is headless and only intended to work with GUIs
explicitly asking for it. This makes it useless to have it in the
autoprobe list.
ao_pulse: work around PulseAudio timing bugs

Work around PulseAudio bugs more effectively. In particular, this
should avoid two issues: playback never finishing at end of file /
segment due to PulseAudio always claiming there's still time before
audio playback reaches the end, and jerky playback especially after
seeking due to bogus output from PulseAudio's timing interpolation
code.

This time, I looked into the PulseAudio code itself and analyzed the
bugs causing problems. Fortunately, two of the serious ones can be
worked around in client code. Write a new get_delay() implementation
doing that, and remove some of the previous workarounds which are now
unnecessary. Also add a pa_stream_trigger() call to ensure playback of
files shorter than prebuf value starts (btw doing that by setting a
low prebuf hits yet another PulseAudio bug, even if you then write the
whole file in one call).

There are still a couple of known PulseAudio bugs that can not be
worked around in client code. Especially, bug 4 below can cause issues
when pausing.

Below is a copy of a message I sent to the pulseaudio-discuss mailing
list, describing some of the PulseAudio bugs:

==================================================

A lot of mplayer2 users with PulseAudio have experienced problems. I
investigated some of those and confirmed that they are caused by
PulseAudio. There are quite a few distinct PulseAudio bugs; some are
analyzed below. Overall, however, I wonder why there are so many fairly
obvious bugs in a widely used piece of software. Is there no
maintenance? Or do people not test it? Some of the bugs are probably
less obvious if you request low latency (though they're not specific to
higher-latency case); do people test the low-latency case only?

1. The timing interpolation functionality can return completely bogus
values for playback position and latency, especially after seeking
(mplayer2 does cork / flush / uncork, as flushing alone does not seem to
remove data already in sink). I've seen quickly repeated seeks report
over 10 second latency, when there aren't any buffers anywhere that big.
I have not investigated the exact cause. Instead I disabled
interpolation and added code to always call
pa_stream_update_timing_info(). (I assume that always waiting for this
to complete, instead of doing custom interpolation, may give bad
performance if it queries a remote server. But at least it works better
locally.)

2. Position/latency reporting is wrong at the end of a stream (after the
lack of more data triggers underflow status). As a result mplayer2 never
ends the playback of a file, as it's waiting forever for audio to finish
playing. The reason for this is that the calculations in PulseAudio add
the whole length of data in the sink to the current latency (subtract
from position), even if the sink does not contain that much data *from
this stream* in underflow conditions. I was able to work around this bug
by calculating latency from pa_timing_info data myself as follows
(ti=pa_timing_info):

    int64_t latency = pa_bytes_to_usec(ti->write_index - ti->read_index, ss);
    latency -= ti->transport_usec;
    int64_t sink_latency = ti->sink_usec;
    if (!ti->playing)
        // this part is missing from PulseAudio itself
        sink_latency -= pa_bytes_to_usec(ti->since_underrun, ss);
    if (sink_latency > 0)
        latency += sink_latency;
    if (latency < 0)
        latency = 0;

However, this still doesn't always work due to the next bug.

3. The since_underrun field in pa_timing_info is wrong if PulseAudio is
resampling the stream. As a result, the above code indicated that the
playback of a 0.1 second 8-bit mono file would take about 0.5 seconds.
This bug is in pa_sink_input_peek(). The problematic parts are:

ilength = pa_resampler_request(i->thread_info.resampler, slength);
...
if (ilength > block_size_max_sink_input)
    ilength = block_size_max_sink_input;
...
pa_memblockq_seek(i->thread_info.render_memblockq, (int64_t) slength, PA_SEEK_RELATIVE, TRUE);
...
i->thread_info.underrun_for += ilength;

This is measuring audio in two different units, bytes for
resampled-to-sink (slength) and original stream (ilength). However, the
block_size_max_sink_input test only adjusts ilength; after that the
values may be out of sync. Thus underrun_for is incremented by less than
it should be to match the slength value used in pa_memblockq_seek.

4. Stream rewind functionality breaks if the sink is suspended (while
the stream is corked). Thus, if you pause for more than 5 seconds
without other audio playing, things are broken after that. The most
obvious symptom is that playback can continue for a significant time
after corking. This is caused by sink_input and sink getting out of
sync. First, after uncorking a stream on a suspended sink,
pa_sink_input_request_rewind() is called while the sink is still in
suspended state. This sets sink_input->thread_info.rewrite_nbytes to -1
and calls pa_sink_request_rewind(). However, the sink ignores rewind
requests while suspended. Thus this particular rewind does nothing. The
problem is that rewrite_nbytes is left at -1. Further calls to
pa_sink_input_request_rewind() do nothing because "nbytes =
PA_MAX(i->thread_info.rewrite_nbytes, nbytes);" sets nbytes to -1, and
the call to pa_sink_request_rewind() is under "if (nbytes != (size_t)
-1) {". Usually, after a sink responds to a rewind request,
rewrite_bytes is reset in pa_sink_input_process_rewind(), but this
doesn't happen if the sink ever ignores one request. This broken state
can be resolved if pa_sink_input_process_rewind() is called due to a
rewind triggered by _another_ stream.

There were more bugs, but I'll leave those for later.