avcodec/proresenc_anatoliy: check against maximum dimensions

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>