Feature unique queue

Made sure that each URI is only crawled once per run.