hevc: check that the VCL NAL types are the same for all slice segments of a frame

Fixes possible invalid memory access for mismatching skipped/non-skipped
slice segments.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Sample-Id: 00001533-google
avcodec/hevc_filter: assert validity of qp predictor input

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avcodec/hevc: propagate error code from hls_coding_quadtree()

Fixes use of uninitialized memory
Fixes out of array read
Fixes: asan_static-oob_123cee5_2630_cov_1869071233_PICSIZE_A_Bossen_1.bin
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
hevc: Consider first quantization group any reference to 0, 0

According to my understanding of T-REC-H.265-2013044 chapter 8.6.1.

Sample-Id: 00001438-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
hevc: Fix modulo operations

Keep qp fields within the range.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/hevc.c
	libavcodec/hevc_filter.c