Revert "Improve decoding quality for lossy wavpack."

This has been implemented more correctly.

This reverts commit a915618a29f3f4197832151a4ed03ccdd585f9cf.
(cherry picked from commit 32e74395a8e88dee1c149aeb36e7a21df431c181)
wavpack: Don't shift minclip/maxclip

Since we are clipping before we shift the values to
16 or 32 bits, we should not shift the min/max clip
values to compensate.

Fixes 8 and 24 bit lossy decoding.

Fixes ticket #871.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 480b133e6f79c470aff0f84d9ed3648d37c32b03)
lavc: add avcodec_is_open().

It allows to check whether an AVCodecContext is open in a documented
way. Right now the undocumented way this check is done in lavf/lavc is
by checking whether AVCodecContext.codec is NULL. However it's desirable
to be able to set AVCodecContext.codec before avcodec_open2().

(cherry picked from commit af08d9aeea870de017139f7b1c44b7d816cf8e56)

Conflicts:

	doc/APIchanges
lavc: make avcodec_close() work properly on unopened codecs.

I.e. free the priv_data and other stuff allocated in
avcodec_alloc_context3() and not segfault.

(cherry picked from commit 0e72ad95f9fef6a6b8ae55e47339a5c40526502f)
lavc: set AVCodecContext.codec in avcodec_get_context_defaults3().

This way, if the AVCodecContext is allocated for a specific codec, the
caller doesn't need to store this codec separately and then pass it
again to avcodec_open2().

It also allows to set codec private options using av_opt_set_* before
opening the codec.
(cherry picked from commit bc901998487bf9b77a423961d9f961bcc28a9291)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
qdm2: Check data block size for bytes to bits overflow.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit dac56d9ce01eb9963f28f26b97a81db5cbd46c1c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
libavcodec: Don't crash in avcodec_encode_audio if time_base isn't set

Earlier, calling avcodec_encode_audio worked fine even if time_base
wasn't set. Now it crashes due to trying to scale the output pts to
the codec context time base. This affects e.g. VLC.

If no time_base is set for audio codecs, set it to the sample
rate.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 9a7dc618c50902e7a171f2deda6430d52c277a95)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
swscale: fix V plane memory location in bilinear/unscaled RGB/YUYV case.

Fixes bug 221.

CC: libav-stable@libav.org
(cherry picked from commit b7542dd3d71d1ee873277020b6a8eab2674bb167)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
h264: disallow constrained intra prediction modes for luma.

Conversion of the luma intra prediction mode to one of the constrained
("alzheimer") ones can happen by crafting special bitstreams, causing
a crash because we'll call a NULL function pointer for 16x16 block intra
prediction, since constrained intra prediction functions are only
implemented for chroma (8x8 blocks).

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 45b7bd7c53b41bc5ff6fc2158831f2b1b1256113)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
ws_snd1: Fix wrong samples count and crash.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9fb7a5af97d8c084c3af2566070d09eae0ab49fc)

Addresses CVE-2012-0848

Reviewed-by: Justin Ruggles <justin.ruggles@gmail.com>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
atrac3: Fix crash in tonal component decoding.

Add a check to avoid writing past the end of the channel_unit.components[]
array.

Bug Found by: cosminamironesei
Fixes CVE-2012-0853
CC: libav-stable@libav.org

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit c509f4f74713b035a06f79cb4d00e708f5226bc5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
shorten: Use separate pointers for the allocated memory for decoded samples.

Fixes invalid free() if any of the buffers are not allocated due to either
not decoding a header or an error prior to allocating all buffers.

Fixes CVE-2012-0858
CC: libav-stable@libav.org

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 204cb29b3c84a74cbcd059d353c70c8bdc567d98)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
golomb: avoid infinite loop on all-zero input (or end of buffer).

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c6643fddba73560f26f90d327c84d8832222a720)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
get_bits: add HAVE_BITS_REMAINING macro.

(cherry picked from commit b44b41633f110e9d938165e0f79c9d32191fc135)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
golomb: use HAVE_BITS_REMAINING() macro to prevent infloop on EOF.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 46b3fbc30b7aaf7fdd52391734cfd6d93af8720a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
flac: fix infinite loops on all-zero input or end-of-stream.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 52e4018be47697a60f4f18f83551766df31f5adf)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
rv20: prevent calling ff_h263_decode_mba() with unset height/width

Prevents a crash of VLC during playback of a invalid matroska file,
found by John Villamil <johnv@matasano.com>.

CC: libav-stable@libav.org
(cherry picked from commit c3e10ae4127c998b809066926a410f40ebd47593)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
wma: Clip WMA1 and WMA2 frame length to 11 bits.

The MDCT buffers in the decoder are only sized for up to 11 bits. The
reverse engineered documentation for WMA1/2 headers say that that for
all samplerates above 32kHz 11 bits are used. 12 and 13 bit support
were added for WMAPro. I was unable to make any Microsoft tools generate
a test file at a samplerate above 48kHz.

Discovered by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit d78bb1a4b2a3a415b68e4e6dd448779eccec64e3)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
aac: fix infinite loop on end-of-frame with sequence of 1-bits.

Based-on-work-by: Ronald S. Bultje <rsbultje@gmail.com>
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1cd9a6154bc1ac1193c703cea980ed21c3e53792)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
matroskadec: Pad AAC extradata.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit d2ee8c17793201ce969afd1f433ba1580c143cd2)

Signed-off-by: Anton Khirnov <anton@khirnov.net>