Add support for nested jobs, caching, and performance enhancements for job status updates

I think you can remove this and just call the private GetAllJobs method you've written below...