scrapy/w3lib — pull request #11 by stav (2013-06-14 17:48:52+00:00)
Title: add_or_replace_parameter can have ? duplicated

If you call `add_or_replace_parameter` with a url with a question mark on the end, it will duplicate it.

>>> add_or_replace_parameter('http://example.com/index.html?', 'page', '1')
'http://example.com/index.html??page=1'

because:

>>> url
'http://example.com/index.html?'

>>> urlparse.urlsplit(url)
SplitResult(scheme='http', netloc='www.pcrichard.com', path='/index.html', query='', fragment='')

>>> bool(query)
False

>>> url.strip().endswith('?')
True