Shutdown ping thread when handler is done