Bind to localhost only when start listening

As we are relying on local URL scheme for testing anyway, there is no need to expose our test server to the world (at least when starting it ourselves).