added back the publish feature and reject requests to robot.txt