Use a slightly more robust mechanism of finding application conf.

Also add a nice reporting message of what the limitations of this approach are.