Add support for nested jobs, caching, and performance enhancements for job status updates