Repository: scrapy/scrapy

Issue:
BrowserLikeContextFactory not available in some conditions
While tracing the error that lead to #2555 I tried the workaround [mentioned in the documentation](https://doc.scrapy.org/en/latest/topics/settings.html#downloader-clientcontextfactory) without success.

This code lives [incontextfactory.py](https://github.com/scrapy/scrapy/blob/c3411373e8a8ee2786588bdad7be469c69a25e2a/scrapy/core/downloader/contextfactory.py#L63) but was not reachable as the import was failing on my system due to #2555.

This file is a large `try/except` block with many potential points of failure and it's likely to trip other users in the future.
That said, could this be refactored to provide a fallback for `BrowserLikeContextFactory` or otherwise reduce the scope of the `try/except` to avoid breaking the API?


Discussion:
I think the try/except was to handle Twisted pre- and post-processing v14.
It can probably be removed to only have the twisted 14+ available imports

Le 13 févr. 2017 7:36 PM, "Renato Alves" <notifications@github.com> a
écrit :

> While tracing the error that lead to #2555
> <https://github.com/scrapy/scrapy/issues/2555> I tried the workaround mentioned
> in the documentation
> <https://doc.scrapy.org/en/latest/topics/settings.html#downloader-clientcontextfactory>
> without success.
>
> This code lives incontextfactory.py
> <https://github.com/scrapy/scrapy/blob/c3411373e8a8ee2786588bdad7be469c69a25e2a/scrapy/core/downloader/contextfactory.py#L63>
> but was not reachable as the import was failing on my system due to #2555
> <https://github.com/scrapy/scrapy/issues/2555>.
>
> This file is a large try/except block with many potential points of
> failure and it's likely to trip other users in the future.
> That said, could this be refactored to provide a fallback for
> BrowserLikeContextFactory or otherwise reduce the scope of the try/except
> to avoid breaking the API?
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/scrapy/scrapy/issues/2560>, or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AA2GGFFoJ1UKP4ao6suB9Szu-U0BYr6Gks5rcKKOgaJpZM4L_kCU>
> .
>


Resolving patch:
diff --git a/scrapy/core/downloader/contextfactory.py b/scrapy/core/downloader/contextfactory.py
--- a/scrapy/core/downloader/contextfactory.py
+++ b/scrapy/core/downloader/contextfactory.py
@@ -1,15 +1,15 @@
 from OpenSSL import SSL
 from twisted.internet.ssl import ClientContextFactory
 
-try:
+from scrapy import twisted_version
+
+if twisted_version >= (14, 0, 0):
 
     from zope.interface.declarations import implementer
 
-    # the following should be available from Twisted 14.0.0
     from twisted.internet.ssl import (optionsForClientTLS,
                                       CertificateOptions,
                                       platformTrust)
-
     from twisted.web.client import BrowserLikePolicyForHTTPS
     from twisted.web.iweb import IPolicyForHTTPS
 
@@ -86,7 +86,7 @@ def creatorForNetloc(self, hostname, port):
                                             'method': self._ssl_method,
                                        })
 
-except ImportError:
+else:
 
     class ScrapyClientContextFactory(ClientContextFactory):
         "A SSL context factory which is more permissive against SSL bugs."


Test patch: