Add support for nested jobs, caching, and performance enhancements for job status updates

Can be rewritten as:

    return _dateTimeService.Now - AllJobsLastUpdate) > TimeSpan.FromMilliseconds(CACHE_INTERVAL_MILLISECONDS