ARQ-1311 NPE when using @Deployment(testable=false)

This PR is based on ARQ-892, so please merge that one first.