avcodec/resample: check for malloc failure

Found-by: CSA
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avcodec/dvdsubdec: free subtitle rectangles if nothing is output

Fixes assertion failure
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>