Resolve 3823 - remove keys from application config

Here's another attempt at this problem. Feels like a clean solution.

Comments please.