Drop unneeded dependency on libevdev

Signed-off-by: Simon Busch <morphis@gravedo.de>