replace remaining PIX_FMT_* flags with AV_PIX_FMT_FLAG_*

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Reduce probing score for flic animation files.

Fixes ticket #2567.