Avoid using wildcards in robots.txt in an effort to get it to work better for CCBot