h264: allow cropping to AVCodecContext.width/height

Override the frame size from the SPS with AVCodecContext values
if the latter specify a size smaller by less than one macroblock.
This is required for correct cropping of MOV files from Canon cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 30f515091c323da59c0f1b533703dedca2f4b95d)

Conflicts:

	libavcodec/h264.c
(cherry picked from commit e1608014c50eeb9f4744a53de0794eb6bb1269a2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit b102d5d97daedb717c023ec7bfa43047d97de284)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
mov: set AVCodecContext.width/height for h264

This is required for correct cropping of files from Canon
cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 8aa93e900449c88c3169ff5636fed03f41779cac)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 2fb4be9a99a2c2a9435339830e3d940171cc0d9b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 0054d70f23edd1f61a10a1c2c687b3a04831feb9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
vc1dec: add flush function for WMV9 and VC-1 decoders

CC: libav-stable@libav.org
(cherry picked from commit 4dc8c8386eef942dba35c4f2fb3210e22b511a5b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 02b72394627933dc8ce26445231a69f00dba491b)

Conflicts:
	libavcodec/vc1dec.c

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 0173a7966b331105158a88f96b9afcc431d2fef8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
nuv: check RTjpeg header for validity

CC: libav-stable@libav.org
(cherry picked from commit 859a579e9bbf47fae2e09494c43bcf813dcb2fad)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 6704522ca9dd32c858ee474492be568c386910f9)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit f31170d4e7f9671e019315391160d454b18d7296)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
imgconvert: avoid undefined left shift in avcodec_find_best_pix_fmt

CC: libav-stable@libav.org
(cherry picked from commit 39bb27bf79bc4c2d8beaed637a14176264cb1916)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 7a7229b52d1900279041991fadbd29b27e8dfe95)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 8812b5f164109553f009ce385e17a1af16b6ea53)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
mpegvideo: Don't use ff_mspel_motion() for vc1

Using ff_mspel_motion assumes that s (a MpegEncContext
poiinter) really is a Wmv2Context.

This fixes crashes in error resilience on vc1/wmv3 videos.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 18f2d5cb9c48d06895960f37467576725c9dc2d1)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit da0c457663479bc1828918e1bb3e4a5e4de0d557)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 899d95efe12f1e250b361837c1c8c06df9ac9b86)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
lavfi: avfilter_merge_formats: handle case where inputs are same

This fixes a double-free crash if lists are the same due to the two
merge_ref() calls at the end of the (useless) merging that happens.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 11b6a82412bcd372adf694a26d83b07d337e1325)

Conflicts:

	libavfilter/formats.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit e5f4e249422834f727bcd432b73af971277f1371)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit b6c5848a1f8fc2755ea70d325acaddae9fac45ab)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
vorbis: Validate that the floor 1 X values contain no duplicates.

Duplicate values in this vector are explicitly banned by the Vorbis I spec
and cause divide-by-zero crashes later on.
(cherry picked from commit ecf79c4d3e8baaf2f303278ef81db6f8407656bc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 9aaaeba45c41cf2b3fa4100abbdee7437428f93c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit d6e250abfc36b239ef0c1fc9d45d588b853bfcb9)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Merge remote-tracking branch 'qatar/release/0.6' into release/0.6

* qatar/release/0.6:
  vorbis: Validate that the floor 1 X values contain no duplicates.
  lavfi: avfilter_merge_formats: handle case where inputs are same
  mpegvideo: Don't use ff_mspel_motion() for vc1
  imgconvert: avoid undefined left shift in avcodec_find_best_pix_fmt
  nuv: check RTjpeg header for validity
  vc1dec: add flush function for WMV9 and VC-1 decoders
  mov: set AVCodecContext.width/height for h264
  h264: allow cropping to AVCodecContext.width/height

Merged-by: Michael Niedermayer <michaelni@gmx.at>
rtmp: fix multiple broken overflow checks

Sanity checks like `data + size >= data_end || data + size < data' are
broken, because `data + size < data' assumes pointer overflow, which is
undefined behavior in C.  Many compilers such as gcc/clang optimize such
checks away.

Use `size < 0 || size >= data_end - data' instead.

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 902cfe2f74d777a7dc20ac68f2393b9f84b790c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
rtmp: fix buffer overflows in ff_amf_tag_contents()

A negative `size' will bypass FFMIN().  In the subsequent memcpy() call,
`size' will be considered as a large positive value, leading to a buffer
overflow.

Change the type of `size' to unsigned int to avoid buffer overflow, and
simplify overflow checks accordingly.

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4e692374f7962ea358c329de38c380103f8991b6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
huffyuvdec: Check init_vlc() return codes.

Prevents out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f67a0d115254461649470452058fa3c28c0df294)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 95ab8d33e1a680f30a5a9605175112008ab81afc)

Conflicts:

	libavcodec/huffyuv.c
(cherry picked from commit 277def59fce10d91e3113e5c0f63e22bc4abfa88)

Conflicts:

	libavcodec/huffyuv.c
(cherry picked from commit adf022f458d75e2c8041262e1906a249366ad518)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
huffyuvdec: Skip len==0 cases

Fixes vlc decoding for hypothetical files that would contain such cases.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0dfc01c2bbf4b71bb56201bc4a393321e15d1b31)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ff41ffeb4cb9ea6df49757dc859619dc3d3ab4f)

Conflicts:

	libavcodec/huffyuv.c
(cherry picked from commit 9bc70fe1ae50fd2faa0b9429d47cfbda01a92ebc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
configure: Make warnings from -Wreturn-type fatal errors

These warnings have no false positives and point to serious bugs.
(cherry picked from commit 99853cb8d4237b810b2fffb4a34f66fd0064ef72)

Conflicts:

	configure

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avcodec/rpza: Perform pointer advance and checks before using the pointers

Fixes out of array accesses
Fixes Ticket2850

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3819db745da2ac7fb3faacb116788c32f4753f34)

Conflicts:

	libavcodec/rpza.c
(cherry picked from commit edba432b8b01d68c22e70a508f47553359f59fb5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
matroska_read_seek: Fix used streams for subtitle index compensation

Might fix Ticket1907 (I have no testcase so i cant test)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4758e32a6c48044f77102a49110c79b4f338f648)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avcodec/dsputil: fix signedness in sizeof() comparissions

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 454a11a1c9c686c78aa97954306fb63453299760)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avcodec/ffv1enc: update buffer check for 16bps

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3728603f1854b5c79d1a64dd3b41b80640ef1e7f)

Conflicts:

	libavcodec/ffv1enc.c
(cherry picked from commit c900c6e5c26cd86cf34f9c8d4347cedbd01f3935)
update for 0.6.7

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>