qdm2: check array index before use, fix out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

(cherry picked from commit a7ee6281f7ef1c29284e3a4cadfe0f227ffde1ed)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
vqavideo: check chunk sizes before reading chunks

Fixes out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab6c9332bfa1e20127a16392a0b85a4aa4840889)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 13093f9767b922661132a3c1f4b5ba2c7338b660)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
roqvideodec: check dimensions validity

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ae610451170cd5a28b33950006ff0bd23036845)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fee26d352a52eb9f7fcd8d9167fb4a5ba015b612)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
dsputil: convert remaining op_pixels_func

Convert to diffptr_t the line_size parameters still int.

Remove all the warnings in dsputil.c
libopencore-amrnb: cosmetics: Group all encoder-related code together

CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
libopencore-amr: Conditionally compile decoder and encoder bits

CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
libopencore-amrwb: Make AMR-WB ifdeffery more precise

The library might provide an encoder in the future, so it's better to
check for the presence of the decoder rather than just the library.

CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
build: The libopencore-amrnb encoder depends on audio_frame_queue

CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
build: make audio_frame_queue a stand-alone component

Encoders requiring it have the dependency expressed in the
configure.
svq3: unbreak decoding

a7d2861d36756b913e85681b86ed3385274e8ced removed necessary braces.
doc: Fix some obsolete references to av* tools as ff* tools

Signed-off-by: Diego Biurrun <diego@biurrun.de>

CC: libav-stable@libav.org
x86: dsputil: int --> ptrdiff_t for ff_put_pixels16_mmxext line_size param

This avoids SIMD-optimized functions having to sign-extend their
line size argument manually to be able to do pointer arithmetic.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
x86: dsputil: Fix h263 loop filter link error in some configurations

This was caused by unconditionally referencing a conditionally compiled
table. Now the code is also compiled conditionally.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
vp56: Remove clear_blocks call, and clear alpha plane U/V DC only

The non-alpha and alpha-Y planes are cleared in the idct_put/add()
calls. For the alpha U/V planes, we only care about the DC for entropy
context prediction purposes, the rest of the data is unused.

Signed-off-by: Martin Storsjö <martin@martin.st>
x86: Use simple nop codes for <= sse (rather than <= mmx)

The "CentaurHauls family 6 model 9 stepping 8" family of CPUs
(flags: fpu vme de pse tsc msr cx8 sep mtrr pge mov pat mmx fxsr sse
up rng rng_en ace ace_en) SIGILLs on long nop codes.

Signed-off-by: Martin Storsjö <martin@martin.st>
h264chroma: Remove duplicate 9/10 bit functions

These functions do the same thing in 16 bit space and don't need
any depth specific clipping.

Signed-off-by: Martin Storsjö <martin@martin.st>
h264: Add add_pixels4/8() to h264dsp, and remove add_pixels4 from dsputil

These functions are mostly H264-specific (the only other user I can
spot is bink), and this allows us to special-case some functionality
for H264. Also remove the 16-bit-coeff with >8bpp versions (unused)
and merge the duplicate 32-bit-coeff for >8bpp (identical).

Signed-off-by: Martin Storsjö <martin@martin.st>
get_bits: Return pointer to buffer that is the result of the alignment

This allows more transparent mixing of get_bits and whole-byte access
without having to touch get_bits internals.

Signed-off-by: Martin Storsjö <martin@martin.st>
h264: Don't store intra pcm samples in h->mb

Instead, keep them in the bitstream buffer until we read them verbatim,
this saves a memcpy() and a subsequent clearing of the target buffer.
decode_cabac+decode_mb for a sample file (CAPM3_Sony_D.jsv) goes from
6121.4 to 6095.5 cycles, i.e. 26 cycles faster.

Signed-off-by: Martin Storsjö <martin@martin.st>
h264/svq3: Stop using draw_edges

Instead, only extend edges on-demand when the motion vector actually
crosses the visible decoded area using ff_emulated_edge_mc(). This
changes decoding time for cathedral from 8.722sec to 8.706sec, i.e.
0.2% faster overall. More generally (VP8 uses this also), low-motion
content gets significant speed improvements, whereas high-motion content
tends to decode in approximately the same time.

Signed-off-by: Martin Storsjö <martin@martin.st>