vaapi: fix inverted condition

How embarrassing. This could make --hwdec=vaapi-copy as well as
screenshots with vo_vaapi randomly fail. Regression since
commit b8382aa.
vaapi: fix non-sense condition

Attempting signed comparison on unsigned value.
vaapi: remove non-VLD entrypoints

These probably don't work. libavcodec doesn't seem to support them,
and neither did the original mplayer-vaapi patch.