wayland-pointer: Remove the conditional around the surface actor too

The actor should always exist.