LPS-42203 Rearrange code - no logic change
LPS-42203 Storing robots.txt if altered