Automate SolidFire Backup to S3 to run in parallel
Channel: scaleoutSean
Date: 2021-04-27

In this video I use PowerShell to run 10 bulk volume jobs (in this case, BulkVolumeRead, which is what Backup to S3 uses) in parallel which makes backup complete faster and without human intervention. 

SolidFire 12.2 has a per-node limit of 8 bulk volume jobs so with 5 nodes theoretically - if all volumes as well as jobs - were perfectly distributed we could run 40 parallel jobs. Twenty parallel jobs is probably still quite unlikely to cause failed jobs, but you could throttle this number lower if it did, or simply reschedule failed jobs.

I use "ForEach-Object -Parallel" in this demo.

For additional context, please see
https://scaleoutsean.github.io/2021/04/21/solidfire-backup-to-s3.html#automating-solidfire-backup-to-s3