make robots environment aware and add route and pages controller to serve robots file for each env