update robots.txt to allow access to wordpress assets