Lower persistence loop updated to use subsetted files