Youtube spider now almost finally finished. Just needs more testing.
Updated pipeline so that it puts the data files were they don't bother,
also updated the file names used to a hash.
Updated for more protection from bans. Disabled cookies and changed user
agent to Google bot.
Makeover of Deviant Art spider. Now it seems to be fully working.
Started working on Ebaumsworld spider
I'm no longer using rope.