implemented jobs pages
robots.txt to prevent dev domains from being indexed