Make sure we handle having a "problems" directory under the application - closes #3184
Merge pull request #3263 from permcody/peacock

Make sure we handle having a "problems" directory under the application