avcodec/truemotion2: use av_calloc() & av_malloc_array()

Signed-off-by: Paul B Mahol <onemda@gmail.com>
avcodec/ratecontrol: give some trivial tips in case of buffer underflows

See Ticket2725

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>