The intention here of looking for a i-1 or i+1 file is that, usually these files appear one after the other on disk. Hence we take a quick look to see if i+1 or i-1 is the conf file for this history file. If not, that's good, we add it to toBeRemovedFileList.add(fileName).

So yes, you are right in thinking that "If you have more than one pair of files that are skipped in the middle, I suspect this would break pretty easily." But, at the end, we iterate through the toBeRemovedList and remove those files. 
The idea of looking at i+1 or i-1 is that, this would be faster than iterating over the entire list of files to be processed and checking each one.