Use case statement in #should_be_visited?
Assignment to local variable unnecessary in
#links_for
Remove unnecessary check if it is an URI

Kernel.URI is doing this by itself
Simplify #url_tracker and #redis
Remove begin block from HTTP#fetch_pages
Minor changes to HTTP#fetch_pages
Simplify PolipusCrawler.crawl
Add AUTHORS
Merge pull request #17 from tmaier/code-style

Minor changes to code style
http gzip content fix
Merge pull request #21 from taganaka/http_compression_test

[WIP] HTTP compression handling
Adding coveralls support
fancy badge added
Merge branch 'coveralls'
Update README.md
Update README.md
Update README.md
some code cleanup
robotex gem included
robots.txt support added