Cache successful paths internally

Fixes #17.

Heh, so I tried this out on help.github.com, a largeish site. Dropped from 5m12s to 4m34s. Going to learn myself some Ruby tooling and try to trim this down a bit more.