Repository: googleapis/google-api-python-client

Issue:
ImportError: No module named _ssl on App Engine since 1.5.0
After updating to 1.5.0, my App running on App Engine stopped working:

```
  File "<...>/base_api_client.py", line 4, in <module>
    from googleapiclient.discovery import build
  File "<...>/lib/googleapiclient/discovery.py", line 63, in <module>
    from googleapiclient.http import BatchHttpRequest
  File "<...>/lib/googleapiclient/http.py", line 39, in <module>
    import ssl
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 97, in <module>
    import _ssl             # if we can't import it, let the error propagate
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 948, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named _ssl
```

The error goes away if I add the ssl module to my app.yaml:

```
- name: ssl
  version: latest
```

However, googleapiclient worked fine previously without using/importing the ssl module, so I am not sure this is intended.

Thanks,



Discussion:
@dhermes any ideas on this one? Seems like oauth2client now requires openssl instead of it being optional.

I don't see any noticeable changes in [`setup.py` history](https://github.com/google/oauth2client/commits/master/setup.py)

I think it's more that maybe some ``try... import... except` clauses were removed?

None of them were "removed" (that I know of), just made more explicit (with actual code definitions moved into their own modules). From the stack trace it looks like `googleapiclient/http.py` is importing `ssl`.

Arg, thanks! Sorry for assuming it was on `oauth2client`.

No worries. Cheers.

Here's the commit that, I think, introduced the error by importing `ssl`: 703c92ca475a5fd3e8cc2c5b3961be0251b5480c

(+@rryk, author of the cited commit)

Let's imagine for a moment that I were a boorish project maintainer incapable of grasping nuance: what's the problem being reported here? [The ssl module is part of the Python standard library, right](https://docs.python.org/2/library/ssl.html)? Why would it be a problem that our library uses the language's standard library?

The problem is that the App Engine documentation states that you can use googleapiclient on App Engine ( https://developers.google.com/api-client-library/python/guide/google_app_engine ) and this is no longer true unless you make special, non-intuitive changes to your app.yaml file, in order to work around some code that may or may not work on App engine anyway (as the commit did not take this use case into account).

@nathanielmanistaatgoogle App Engine used a heavily modified version of Python, which does not have ssl included by default. Users have to make a change to their app.yaml to opt-in to ssl. We can document this, but IMO it should remain optional as it was before.

@nabla-c0d3: thank you for your strong and clear response.

@jonparrott: elaborate a bit more on your opinion? Aside from gumbling in App Engine's general direction I'm not yet sure what the right thing to do here is. I'm reluctant to take on a special requirement for the library like "must not import ssl" that would be awkward or weird to verify in tests.

@nathanielmanistaatgoogle grumbling in App Engine's direction is deserved, I think. I think we have two options here:
1. Be okay with the fact that we broke users in an environment that previously worked without additional config, and thus shift the burden onto our users.
2. Add import guards and unbreak users in that environment.

We can grumble at App Engine either way, it's just what we want to do between now and whenever I can convince the runtime team to turn on ssl by default.

A guarded import would wound my pride but I think I might survive. Is there an issue in a public issue tracker about turning on ssl by default to which we might link our TODO?

@rryk, any opinion on any of this?

Public issue tracker for App engine is in limbo. I'm part of the runtime team, so I will bring it up in our meeting tomorrow.

Sorry for insisting but I just want to re-iterate something: I don't think the newly introduced code that is using `ssl` would even work on App Engine. 
App Engine's HTTP stack (httplib, urllib2 and all use it) is heavily customized and I would not expect it to throw an `ssl.SSLError` ever. I am not 100% sure and the App Engine team would know better of course, but I don't this issue is about enabling `ssl` on App Engine. It's about Cython-specific code (ie. code that doesn't work on App Engine) that was introduced in the commit.

@nabla-c0d3: are you sure you're not conflating [CPython](https://en.wikipedia.org/wiki/CPython) and [Cython](http://cython.org/)? They are very different things.

I'm not following your point, though - are you saying that the version of the `ssl` module made available on App Engine does not have the same API as the standard library's `ssl` module?

@nathanielmanistaatgoogle yeah I meant CPython. 

The `ssl` modules are probably the same but when making HTTP requests on App Engine, the URL Fetch service is used (https://cloud.google.com/appengine/docs/python/urlfetch/) behind the scenes even when using standard libraries like httplib or urllib2. It is, I think, a completely different implementation which would never throw an `ssl.SSLError` (what the commit tries to handle) since it probably doesn't even use Python's `ssl` module at all.

The `ssl` module is also a recent addition to App Engine and because it is optional, I would not expect the URL Fetch service to throw an `ssl.SSLError` when something fails at the SSL level; hence the code that was pushed most likely does not work on App Engine.

Hi, I am on vacation and don't have a good internet access or a computer to
make any changes. Import guard sounds like an acceptable solution to me
though.

On Wed, Feb 24, 2016, 14:51 Alban Diquet notifications@github.com wrote:

> @nathanielmanistaatgoogle https://github.com/nathanielmanistaatgoogle
> yeah I meant CPython.
> 
> The ssl modules are probably the same but when making HTTP requests on
> App Engine, the URL Fetch service is used (
> https://cloud.google.com/appengine/docs/python/urlfetch/) behind the
> scenes (even when using standard libraries like httplib or urllib2). It is,
> I think, a completely different implementation which would never throw an
> ssl.SSLError (what the commit tries to handle) since it probably doesn't
> even use Python's ssl module at all.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/google/google-api-python-client/issues/191#issuecomment-188006072
> .


Resolving patch:
diff --git a/googleapiclient/http.py b/googleapiclient/http.py
--- a/googleapiclient/http.py
+++ b/googleapiclient/http.py
@@ -38,11 +38,18 @@
 import os
 import random
 import socket
-import ssl
 import sys
 import time
 import uuid
 
+# TODO(issue 221): Remove this conditional import jibbajabba.
+try:
+  import ssl
+except ImportError:
+  _ssl_SSLError = object()
+else:
+  _ssl_SSLError = ssl.SSLError
+
 from email.generator import Generator
 from email.mime.multipart import MIMEMultipart
 from email.mime.nonmultipart import MIMENonMultipart
@@ -146,7 +153,7 @@ def _retry_request(http, num_retries, req_type, sleep, rand, uri, method, *args,
       exception = None
       resp, content = http.request(uri, method, *args, **kwargs)
     # Retry on SSL errors and socket timeout errors.
-    except ssl.SSLError as ssl_error:
+    except _ssl_SSLError as ssl_error:
       exception = ssl_error
     except socket.error as socket_error:
       # errno's contents differ by platform, so we have to match by name.


Test patch: