Fetch config and limits before the queue
Make app-only auth work

With code from @freynolds

Closes #3