[#5] Moved web crawler configs out of apache subdirectories since these are not specific to Apache configuration.
[#5] Added Google crawler file for live server.