Added ability to scan S3 folder for subfolders, and process those in the job