IMPROVEMENT: AccessPoint stops adpatively redirecting on connection timeouts
DuplicateHashFilter: track last 3 duplicate hashes
ROBOTS: Huge improvement in slow N^2 function that checked every path against every robot rule,
now check canonicalized path only once!
up compiler to 1.6!