avcodec/jpeg2000dec: Port tile size check from j2kdec

Fixes potential integer overflow

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
j2k & jpeg2000dec: reset numX/Ytiles if tiles failed allocation

Fixes null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>