avfilter/vf_hqx: avoid floats

this likely fixes fate failures

2748170880 -> 5389024880 dezicycles

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avfilter/vf_hqx: optimize table init

5389024880 -> 1389386610 dezicycles

This surely can be optimized more, i just didnt want to cause a slowdown
when trying to make the fate test bitexact.
Further optimization left to ubitux

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>