demux_mkv: reject 0 TimecodeScale

Also reject anything over INT_MAX; no particular reason for this upper
bound.

Fixes #1317.