PY3: use six for robotparser and urlparse

Not very sure about the text encoding, but it should at least work for current python 2.x environments.