Updated to accomodate new Heritrix revisit handling