jpeg2k: Validate resolution levels

There are 32 maximum decomposition levels, thus 33 resolution levels.
jpeg2k: Check cblk size

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
jpeg2k: define the maximum decomposition levels

And define the resolution levels according.
jpeg2k: Clean up return paths and error messages
jpeg2k: Use bytestream2

Prevent a number of overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
jpeg2k: Validate sot parsing

Avoid some overreads.
jpeg2k: Validate siz parsing

Check the number components and tile dimensions and number.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
jpeg2k: Validate block lengthinc

Currently we are using an array with a static data size.

Similar to a patch with the same purpose by Michael Niedermayer.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
jpeg2k: Check nonzerobits validity

Prevent integer overflows.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
jpeg2k: Check component number in get_coc() and get_qcc()

Avoid overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
jpeg2k: check len before parsing header

Avoid overread.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
jpeg2k: Correctly calculate sgnd

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
jpeg2k: remove an unnecessary field in Jpeg2KCodingStyle

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
drop: jpeg2000: Fix Unintended sign extension in malloc arguments of cblk.

Those variables could go away.

Fixes CID1026770
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
drop: jpeg2000: Fix unintended sign extension in malloc arguments of prec

The variables could be collated as well.

Fixes: CID1026772/3
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
jpeg2k: Check that nreslevels2decode has been initialized before use

Avoid buffer overread.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
jpeg2k: Propagate error code from get_cox()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
jpeg2k: remove unneeded returns
jpeg2k: Check that theres a SOT before SOD

Avoid overread.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
jpeg2k: check that we have enough components for MCT

Avoid overread.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>