avcodec/dvbsubdec: Cleanup on *malloc failure

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>