Cc 883

Created for code review. If you want to see my rationale for making these changes, that is in the last few comments in the following Jira ticket:
https://seomoz.atlassian.net/browse/CC-883

Not only that, strictly adhering to RFC 3986 also solves the issues with query-string normalizing that had been bedeviling me. (In other words, custom crawler was at fault for performing illegal normalization when it turned "url=http://..." into "url=http%3A%2F%2F..." in the query string.) That's in addition to the illegal path normalizing which prompted ticket CC-883.