More cleanup changes

The pull request is not totally clean, but the idea is there.

cleanup() needs some cleaning so it doesn't crash later for no reason. This one fixes the ViewAdapter removal crash and alos some rare crash where onSingleTapConfirmed is called after the view is gone.

I think the code is simple and obvious enough to be integrated.