Enable robots and alter robots.txt by domain
Merge branch '2673-field-change' into 2673-robots
Add Support for changing robots.txt
Add robotstxt to make file
Merge branch '2673-field-change' into 2673-robots
Merge pull request #2728 from openscholar/2673-robots

Add Robots.txt altering for unlisted sites