avcodec/vmnc: Check  that rectangles are within the picture

Prevents out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ba02602aa7fc7d38db582e75b8b093fb3c1608d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avcodec/jpeg2000dec: fix error detection in pix_fmt_match()

Fixes out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8001e9f7d17e90b4b0898ba64e3b8bbd716c513c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>