Properly handle notbuilt jenkins jobs.