Issue 1:
when I request http://tjw_161130192022480.company.qihuiwang.com/

return this: invalid hostname: tjw_161130192022480.company.qihuiwang.com,

I don't know how to solve this question !

hlep me !

Issue 2:
I have url with invalid hostname - it does not match IDNA standards. Scrapy fails with that.

scrapy fetch "https://mediaworld_it_api2.frosmo.com/?method=products&products=[%22747190%22]"

2018-07-06 11:53:09 [scrapy.core.scraper] ERROR: Error downloading <GET https://mediaworld_it_api2.frosmo.com/?method=products&products=[%22747190%22]>
Traceback (most recent call last):
File "/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "/home/pawel/scrapy/scrapy/core/downloader/middleware.py", line 43, in process_request
defer.returnValue((yield download_func(request=request,spider=spider)))
File "/home/pawel//scrapy/scrapy/utils/defer.py", line 45, in mustbe_deferred
result = f(*args, **kw)
File "/home/pawel/scrapy/scrapy/core/downloader/handlers/__init__.py", line 65, in download_request
return handler.download_request(request, spider)
File "/home/pawel/scrapy/scrapy/core/downloader/handlers/http11.py", line 67, in download_request
return agent.download_request(request)
File "/home/pawelscrapy/scrapy/core/downloader/handlers/http11.py", line 331, in download_request
method, to_bytes(url, encoding='ascii'), headers, bodyproducer)
File "/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/twisted/web/client.py", line 1649, in request
endpoint = self._getEndpoint(parsedURI)
File "/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/twisted/web/client.py", line 1633, in _getEndpoint
return self._endpointFactory.endpointForURI(uri)
File "/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/twisted/web/client.py", line 1510, in endpointForURI
uri.port)
File "/home/pawel/scrapy/scrapy/core/downloader/contextfactory.py", line 59, in creatorForNetloc
return ScrapyClientTLSOptions(hostname.decode("ascii"), self.getContext())
File "/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/twisted/internet/_sslverify.py", line 1152, in __init__
self._hostnameBytes = _idnaBytes(hostname)
File "/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/twisted/internet/_idna.py", line 30, in _idnaBytes
return idna.encode(text)
File "/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/idna/core.py", line 355, in encode
result.append(alabel(label))
File "/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/idna/core.py", line 265, in alabel
raise IDNAError('The label {0} is not a valid A-label'.format(label))
IDNAError: The label mediaworld_it_api2 is not a valid A-label

IDNA error is legitmate. This url https://mediaworld_it_api2.frosmo.com/?method=products&products=[%22747190%22] is not valid according to IDNA standard.

In [1]: x = "https://mediaworld_it_api2.frosmo.com/?method=products&products=[%22747190%22]"

In [2]: import idna

In [3]: idna.encode(x)
---------------------------------------------------------------------------
IDNAError Traceback (most recent call last)
<ipython-input-3-c97070e17b57> in <module>()
----> 1 idna.encode(x)

/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/idna/core.pyc in encode(s, strict, uts46, std3_rules, transitional)
353 trailing_dot = True
354 for label in labels:
--> 355 result.append(alabel(label))
356 if trailing_dot:
357 result.append(b'')

/home/pawel/.virtualenvs/scrapy/local/lib/python2.7/site-packages/idna/core.pyc in alabel(label)
263 ulabel(label)
264 except IDNAError:
--> 265 raise IDNAError('The label {0} is not a valid A-label'.format(label))
266 if not valid_label_length(label):
267 raise IDNAError('Label too long')

IDNAError: The label https://mediaworld_it_api2 is not a valid A-label

How should scrapy handle this url? Should we download it regardless of validity?