Trim trailing / from urls and added a user agent header to request.
Removed debug output.