Revert "[PD-569] updated robots.txt per Seth's request."
Merge pull request #961 from DirectEmployers/revert-954-pd-569

Revert "[PD-569] updated robots.txt per Seth's request."