I see '/'.join in many places of this code. Maybe it would be good idea to change this in favor of python's urljoin from urlparse?