Make crawlers ignore the 'temp' directory