initial workpiece
getting the page encoding
gitignore added
decoding pages right after download
page caching
url processing and word trimming
primitive thread pool
forgotten PageProcessor.hs
visit urls only once
taking relative paths into account
tackling dot paths
probing content types, fixed new links queueing
logging facility
better relative paths handling
proper worker killing on exception
building global base and running queries across it
sketch of a main
relative urls fixes
less verbose http error messages
limit number of pages to analyze