Remove uneccessary html files, ban all spiders and move lebowski.json to the db folder