Added some calls to purell.NormalizeURL to NormalizeUrl's.
Merge branch 'master' into NormalizeUrl
Remove the Fragment from all parsed URLs before processing.
NormalizeUrl added tests for SID, case, and SIDs.
Merge branch 'master' into NormalizeUrl
This patch makes NormalizeUrl pass all of test.sh.
NormalizeUrl: sort the query keys to make solid the param order.
Merge branch 'master' into NormalizeUrl
Merge branch 'master' into NormalizeUrl
SubdomRobots checkpoint needed as I'll end my day very soon.
Rough draft of the SubdomRobots feature. Lots of test fails at this
point.
Fixed bug with robotstxt.Test. SubdomRobots now passes existing tests.
Merge branch 'master' into SubdomRobots
Added tests to verify SubdomRobots is working correctly.
Clean up of docs for SubdomRobots patch.
First pass at ContentSize implementation.
Change NormalizeUrl branch to be case-insensitive wrt SID.
Added ContentSize tests.
Merge branch 'master' into ContentSize
Use Content-Length to grow bytes buffer in ContentSize branch.