avfilter/vf_rotate: increase fixed point precision

This ensures int_sins output doesnt have "random" lsbs

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avfilter/vf_rotate: fix location of update operation

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avfilter/vf_rotate: fix several of by 1 errors

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avfilter/vf_rotate: make int*90° rotates suck less speedwise

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>