Don't unnecessarily stop Poll handles

It hurts perfornamce, specially on Windows