Adding robotstxt and googleverification to register the webpage in google crawler