demux_mkv: fix a possible out of bounds access

The if branch has a weak check to test whether the codec_id is the short
ID, and handles the long IDs in the else branch. The long IDs are all
longer than 12 bytes long, so hardcoding the string offset to get the
trailing part of the name makes sense. But the if condition checks for
another thing, which could get the else branch run even if the codec_id
is short.

Fix the bogus control flow and check if the codec_id is long enough. One
of these checks could be considered redundant, but include them both for
defensive coding.
demux_mkv: cosmetics
demux_mkv: simplify extradata handling

It was more complicated than necessary.

The behavior changes slightly. Now it might pass through extradata when
it didn't before (hopefully harmless), and doesn't fail with an error if
extradata is not available, even though it's needed (harmless, will fail
either way).
subprocess-win: remove a Vista-only flag

PIPE_REJECT_REMOTE_CLIENTS isn't supported on XP. It's not really
needed, so remove it. Also fix error checking for CreateNamedPipe.