Stop server, but only if we started it