Do not rely on a play app running to gather metrics

Fixes #44