Defer server creation

This gives the application time to load and make any needed
configuration changes.
Make the caller responsible for casting
Improve service directory logging