crawler add active links to queue and wait for links to activate when there is none