h264: fix ff_generate_sliding_window_mmcos() prototype.

It's been returning an error value since
bad446e251405dc250c3cbee199072e083a1e4b9

Also check for the errors it returns.
(cherry picked from commit ea382767ad2191acbe97e90624059723e15f0e4b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avf_showwaves: fix null pointer dereference

Found-by: durandal_1707
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e4eebc2da9da886e1bdf87d29e9a4c5b55111036)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
h264/ff_generate_sliding_window_mmcos: fix use of uninitialized variable

Fixes CID968587
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fba3e79a3e46d7611817f9051b33885eb1ff962)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
h264: print mismatching opcode details in check_opcodes()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dcbe15813ed09cf491e75a21cce0e751f5bc2b34)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
h264: fix use of uninitialized variable

Fixed CID968588

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc86f88b32a288b252a088ee3c77b4f6219d54d5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Update for 1.0.3

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
matroskaenc: add codec_tag lists back.

This reverts 312645e :
"Do not set codec_tag property for matroska muxers."

Also adds dummy codec_tag lists with codecs
supported in mkv but not in wav / avi.

Fixes ticket #2169.
(cherry picked from commit df39c3ce385c02cbd8046298578ea7454c0a0f81)
wtvdec: demux thumbnail picture to AVStream.attached_pic

Fixes ticket #2133.

(cherry picked from commit 508836932fcbc3c109fcc4df7a2f2b0bf21bbece)
doc/muxers.texi: Fix mp3 picture attachment documentation.
(cherry picked from commit 99eedfc40086972987aa27df8b1259c8bf15b20c)
Do not change codec in flv streams if the user has forced a codec.

Fixes ticket #2218.
(cherry picked from commit 6a50e8a190bc1329fd3e76e8497bb7f870b6e69b)
movtextenc: fix pointer messup and out of array accesses

Fixes Ticket2213

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b0635e2fcf80717dd618ef75d3317d62ed85c300)
dvenc: dont fail hard if the timecode is invalid

Instead just dont store the timecode
Fixes Ticket2187

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f0eacbc760c8b27c2064cea5395e32971853c73c)
ffmpeg: dont allow -flags to override -pass

Fixes Ticket2154

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ccf9dd00da055e94117b56cead4af80ff331b00e)
lavf/mov: skip version and flags attributes in mov_read_chan function

Fixes ticket #1764.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59d40fc7e66e6ebb2d89e25115492bfe13b07d95)
sanm: check image dimensions before using them

Avoids integer overflows and out of array accesses.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 49b729d3af8464de431362e6c5b3027102bc2f88)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
rtpenc: fix overflow checking in avc_mp4_find_startcode()

The check `start + res < start' is broken since pointer overflow is
undefined behavior in C.  Many compilers such as gcc/clang optimize
away this check.

Use `res > end - start' instead.  Also change `res' to unsigned int
to avoid signed left-shift overflow.

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2f014567cfd63e58156f60666f1a61ba147276ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
rtmp: fix multiple broken overflow checks

Sanity checks like `data + size >= data_end || data + size < data' are
broken, because `data + size < data' assumes pointer overflow, which is
undefined behavior in C.  Many compilers such as gcc/clang optimize such
checks away.

Use `size < 0 || size >= data_end - data' instead.

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 902cfe2f74d777a7dc20ac68f2393b9f84b790c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
rtmp: fix buffer overflows in ff_amf_tag_contents()

A negative `size' will bypass FFMIN().  In the subsequent memcpy() call,
`size' will be considered as a large positive value, leading to a buffer
overflow.

Change the type of `size' to unsigned int to avoid buffer overflow, and
simplify overflow checks accordingly.

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4e692374f7962ea358c329de38c380103f8991b6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
huffyuvdec: Check init_vlc() return codes.

Prevents out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f67a0d115254461649470452058fa3c28c0df294)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
huffyuvdec: Skip len==0 cases

Fixes vlc decoding for hypothetical files that would contain such cases.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0dfc01c2bbf4b71bb56201bc4a393321e15d1b31)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>