vmdav: return meaningful error codes

Signed-off-by: Paul B Mahol <onemda@gmail.com>
vmdav: unbreak decoding of samples from game The Last Dynasty

This fixes video output with samples HG060808.VMD and
HG060810.VMD. Regression since c0cbe36b18ab3e.

While here show warning if decoding is aborted for some reason.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
avutil/mem: add av_memdup()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>