avcodec/asvenc: Fix integer overflow in level

Warn if the qscale is too low for the input data and clip levels to
minimize artifacts

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avcodec/asvenc: fix AAN scaling

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>