fix robots.txt directives array logic #2797
Merge pull request #3110 from openscholar/2797-qa

fix robots.txt directives array logic #2797