Initial, woo\!
Added recursive URL search
Removes duplicates before printing
Moved around sorting code, tweaked it, and handled some null scans of sites.
Added support for single-level directory traversal (..) in URLs.
Added readme back in
Changed readme name
Merge branch 'master' of https://github.com/rsmoz/robCrawl