Moving from svn to git
Uploaded the distributed version to the repo afterall
Commited the deletion of unused files
Improved Serialization of the objects. There is still improvement possible: Checkout non Java API serizalizations
removed jsoup but added tika instead.
Managed to get things running with streams and I have to say it works very well.
Modified the locking on the livedatastores. and simplified the logic in the crawling process
Modified the locking on the livedatastores. and simplified the logic in the crawling process
Added OutputStream (not finished yet) and String page processing
Reverted the outputstream stuff and added the possibility to write to an outputstream from the AbstractPage
Worked on blocking stream reading
Implemented page redirect
Started implementing crawling with disk persistent frontier
Merge branch 'master' of https://github.com/Esquive/iticrawler

Moved from bitbucket to github.