disallowed the premium registration pages via robots.txt