mkv: fix declaration/definition mismatch of demux_sys_t

Kill many compiler warnings.