Avoid leaking the provider before ctor finished

To avoid a race, make sure we don't leak the instance of the Provider
before its constructor is finished.