avfilter/vf_deshake: check rx

Fixes crash on SSE2
Fixes Ticket2443

Note, its possible to restructure the code to avoid the 16pixel limitation,
and such change is welcome!

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>