Skip job execution if app is disabled

Backport of 47a6d70c30680a30c56f4855f579692b4ef7f786 from master