Repository: googleapis/google-api-python-client

Issue:
Dependencies fail to resolve correctly when installing via egg
#### Environment details

  - OS type and version: CentOS 7 / 8
  - Python version: `Python 3.8.5` and `Python 3.8.3`
  - pip version: `pip 20.1.1` and `pip 19.3.1`, respectively with the Python versions
  - `google-api-python-client` version: `1.12.8` (latest)

#### Steps to reproduce

  1. Create this minimal setup.py:
  ```
from setuptools import setup, find_packages

setup(
    name='FooModule',
    version='0.0.1',
    author='Foo Bar',
    author_email='author@example.com',
    description='foo',
    packages=find_packages(),
    install_requires=['google-api-python-client'],
)
```

  2. Ensure `wheel` and `setuptools` are installed.

```
[root@f369e3b27dbf foo_module]# pip3 list installed
Package    Version
---------- -------
pip        19.3.1
setuptools 41.6.0
wheel      0.36.2
```

  3. Build `FooModule` as an egg: `python3 setup.py bdist_egg`

  4. Try to install the egg: `python3 -m easy_install dist/FooModule-0.0.1-py3.8.egg` OR `python3 setup.py install`. This should fail! The error is `error: google-auth 2.0.0.dev0 is installed but google-auth<2.0dev,>=1.21.1 is required by {'google-api-core'}` See the full output below.
<details>
  <summary>Click to expand</summary>

```
[root@f369e3b27dbf foo_module]# python3 setup.py install
running install
running bdist_egg
running egg_info
writing FooModule.egg-info/PKG-INFO
writing dependency_links to FooModule.egg-info/dependency_links.txt
writing requirements to FooModule.egg-info/requires.txt
writing top-level names to FooModule.egg-info/top_level.txt
reading manifest file 'FooModule.egg-info/SOURCES.txt'
writing manifest file 'FooModule.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to install

creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying FooModule.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying FooModule.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying FooModule.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying FooModule.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying FooModule.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/FooModule-0.0.1-py3.8.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing FooModule-0.0.1-py3.8.egg
Copying FooModule-0.0.1-py3.8.egg to /usr/local/lib/python3.8/site-packages
Adding FooModule 0.0.1 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/FooModule-0.0.1-py3.8.egg
Processing dependencies for FooModule==0.0.1
Searching for google-api-python-client
Reading https://pypi.org/simple/google-api-python-client/
Downloading https://files.pythonhosted.org/packages/83/fc/98045b8c5e0ba12929d423e9ff6b742951bb846707539b18f19b27c6ddc3/google_api_python_client-1.12.8-py2.py3-none-any.whl#sha256=3c4c4ca46b5c21196bec7ee93453443e477d82cbfa79234d1ce0645f81170eaf
Best match: google-api-python-client 1.12.8
Processing google_api_python_client-1.12.8-py2.py3-none-any.whl
Installing google_api_python_client-1.12.8-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/google_api_python_client-1.12.8-py3.8.egg/EGG-INFO/requires.txt
Adding google-api-python-client 1.12.8 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/google_api_python_client-1.12.8-py3.8.egg
Searching for uritemplate<4dev,>=3.0.0
Reading https://pypi.org/simple/uritemplate/
Downloading https://files.pythonhosted.org/packages/bf/0c/60d82c077998feb631608dca3cc1fe19ac074e772bf0c24cf409b977b815/uritemplate-3.0.1-py2.py3-none-any.whl#sha256=07620c3f3f8eed1f12600845892b0e036a2420acf513c53f7de0abd911a5894f
Best match: uritemplate 3.0.1
Processing uritemplate-3.0.1-py2.py3-none-any.whl
Installing uritemplate-3.0.1-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
Adding uritemplate 3.0.1 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/uritemplate-3.0.1-py3.8.egg
Searching for six<2dev,>=1.13.0
Reading https://pypi.org/simple/six/
Downloading https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl#sha256=8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced
Best match: six 1.15.0
Processing six-1.15.0-py2.py3-none-any.whl
Installing six-1.15.0-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
Adding six 1.15.0 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/six-1.15.0-py3.8.egg
Searching for httplib2<1dev,>=0.15.0
Reading https://pypi.org/simple/httplib2/
Downloading https://files.pythonhosted.org/packages/15/7e/51e5bd333c0afa1c7bdbf98eb3b0ccf5167e2b1ecc8b4d13e9cc29291f81/httplib2-0.19.0-py3-none-any.whl#sha256=749c32603f9bf16c1277f59531d502e8f1c2ca19901ae653b49c4ed698f0820e
Best match: httplib2 0.19.0
Processing httplib2-0.19.0-py3-none-any.whl
Installing httplib2-0.19.0-py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/httplib2-0.19.0-py3.8.egg/EGG-INFO/requires.txt
Adding httplib2 0.19.0 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/httplib2-0.19.0-py3.8.egg
Searching for google-auth>=1.16.0
Reading https://pypi.org/simple/google-auth/
Downloading https://files.pythonhosted.org/packages/68/c3/6851ad3e029ac11e4beb6470b9465f4affce345cba8ef7cbf7fbd647c89a/google_auth-2.0.0.dev0-py2.py3-none-any.whl#sha256=50eda7bbeaed36ad17a87a94a770b9ec97e9a64add701942a4cd4dcbe291f3df
Best match: google-auth 2.0.0.dev0
Processing google_auth-2.0.0.dev0-py2.py3-none-any.whl
Installing google_auth-2.0.0.dev0-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/google_auth-2.0.0.dev0-py3.8.egg/EGG-INFO/requires.txt
Adding google-auth 2.0.0.dev0 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/google_auth-2.0.0.dev0-py3.8.egg
Searching for google-auth-httplib2>=0.0.3
Reading https://pypi.org/simple/google-auth-httplib2/
Downloading https://files.pythonhosted.org/packages/bd/4e/992849016f8b0c27fb604aafd0a7a724db16128906197bd1245c6f18e6a1/google_auth_httplib2-0.0.4-py2.py3-none-any.whl#sha256=aeaff501738b289717fac1980db9711d77908a6c227f60e4aa1923410b43e2ee
Best match: google-auth-httplib2 0.0.4
Processing google_auth_httplib2-0.0.4-py2.py3-none-any.whl
Installing google_auth_httplib2-0.0.4-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/google_auth_httplib2-0.0.4-py3.8.egg/EGG-INFO/requires.txt
Adding google-auth-httplib2 0.0.4 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/google_auth_httplib2-0.0.4-py3.8.egg
Searching for google-api-core<2dev,>=1.21.0
Reading https://pypi.org/simple/google-api-core/
Downloading https://files.pythonhosted.org/packages/c8/3f/f7faa8a96408167c03d6fe07e2bee9b383ec18a7a1117d32912eb0c1b78f/google_api_core-1.26.0-py2.py3-none-any.whl#sha256=002e44c533299aecd9dd265d200f9eacd9957cddd2c72e2cd1cb5cea127e972d
Best match: google-api-core 1.26.0
Processing google_api_core-1.26.0-py2.py3-none-any.whl
Installing google_api_core-1.26.0-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/google_api_core-1.26.0-py3.8.egg/EGG-INFO/requires.txt
Adding google-api-core 1.26.0 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/google_api_core-1.26.0-py3.8.egg
Searching for pyparsing<3,>=2.4.2
Reading https://pypi.org/simple/pyparsing/
Downloading https://files.pythonhosted.org/packages/8a/bb/488841f56197b13700afd5658fc279a2025a39e22449b7cf29864669b15d/pyparsing-2.4.7-py2.py3-none-any.whl#sha256=ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b
Best match: pyparsing 2.4.7
Processing pyparsing-2.4.7-py2.py3-none-any.whl
Installing pyparsing-2.4.7-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
Adding pyparsing 2.4.7 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/pyparsing-2.4.7-py3.8.egg
Searching for rsa<5,>=3.1.4
Reading https://pypi.org/simple/rsa/
Downloading https://files.pythonhosted.org/packages/bf/87/dc7a6ebf0afbc602548627fa48e9c1147fa187233bf71d4c51c76a2cfb27/rsa-4.7-py3-none-any.whl#sha256=a8774e55b59fd9fc893b0d05e9bfc6f47081f46ff5b46f39ccf24631b7be356b
Best match: rsa 4.7
Processing rsa-4.7-py3-none-any.whl
Installing rsa-4.7-py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/rsa-4.7-py3.8.egg/EGG-INFO/requires.txt
Adding rsa 4.7 to easy-install.pth file
Installing pyrsa-decrypt script to /usr/local/bin
Installing pyrsa-encrypt script to /usr/local/bin
Installing pyrsa-keygen script to /usr/local/bin
Installing pyrsa-priv2pub script to /usr/local/bin
Installing pyrsa-sign script to /usr/local/bin
Installing pyrsa-verify script to /usr/local/bin

Installed /usr/local/lib/python3.8/site-packages/rsa-4.7-py3.8.egg
Searching for pyasn1-modules>=0.2.1
Reading https://pypi.org/simple/pyasn1-modules/
Downloading https://files.pythonhosted.org/packages/95/de/214830a981892a3e286c3794f41ae67a4495df1108c3da8a9f62159b9a9d/pyasn1_modules-0.2.8-py2.py3-none-any.whl#sha256=a50b808ffeb97cb3601dd25981f6b016cbb3d31fbf57a8b8a87428e6158d0c74
Best match: pyasn1-modules 0.2.8
Processing pyasn1_modules-0.2.8-py2.py3-none-any.whl
Installing pyasn1_modules-0.2.8-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/pyasn1_modules-0.2.8-py3.8.egg/EGG-INFO/requires.txt
Adding pyasn1-modules 0.2.8 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/pyasn1_modules-0.2.8-py3.8.egg
Searching for cachetools<5.0,>=2.0.0
Reading https://pypi.org/simple/cachetools/
Downloading https://files.pythonhosted.org/packages/bb/72/8df2e0dc991f1a1d2c6869404e7622e8ee50d80bff357dbb57c3df70305b/cachetools-4.2.1-py3-none-any.whl#sha256=1d9d5f567be80f7c07d765e21b814326d78c61eb0c3a637dffc0e5d1796cb2e2
Best match: cachetools 4.2.1
Processing cachetools-4.2.1-py3-none-any.whl
Installing cachetools-4.2.1-py3-none-any.whl to /usr/local/lib/python3.8/site-packages
Adding cachetools 4.2.1 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/cachetools-4.2.1-py3.8.egg
Searching for requests<3.0.0dev,>=2.18.0
Reading https://pypi.org/simple/requests/
Downloading https://files.pythonhosted.org/packages/29/c1/24814557f1d22c56d50280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl#sha256=c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e
Best match: requests 2.25.1
Processing requests-2.25.1-py2.py3-none-any.whl
Installing requests-2.25.1-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/requests-2.25.1-py3.8.egg/EGG-INFO/requires.txt
Adding requests 2.25.1 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/requests-2.25.1-py3.8.egg
Searching for pytz
Reading https://pypi.org/simple/pytz/
Downloading https://files.pythonhosted.org/packages/70/94/784178ca5dd892a98f113cdd923372024dc04b8d40abe77ca76b5fb90ca6/pytz-2021.1-py2.py3-none-any.whl#sha256=eb10ce3e7736052ed3623d49975ce333bcd712c7bb19a58b9e2089d4057d0798
Best match: pytz 2021.1
Processing pytz-2021.1-py2.py3-none-any.whl
Installing pytz-2021.1-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
Adding pytz 2021.1 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/pytz-2021.1-py3.8.egg
Searching for protobuf>=3.12.0
Reading https://pypi.org/simple/protobuf/
Downloading https://files.pythonhosted.org/packages/b7/d3/37c36b3f5b125ac0d5b9f10d51dbecca355e24c233fd8e447c9879338ec2/protobuf-4.0.0rc2-py2.py3-none-any.whl#sha256=5df64b064b1b15e03768295939cbb5595733a02e8945dc033e56c6a9667a225c
Best match: protobuf 4.0.0rc2
Processing protobuf-4.0.0rc2-py2.py3-none-any.whl
Installing protobuf-4.0.0rc2-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/protobuf-4.0.0rc2-py3.8.egg/EGG-INFO/requires.txt
Adding protobuf 4.0.0rc2 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/protobuf-4.0.0rc2-py3.8.egg
Searching for packaging>=14.3
Reading https://pypi.org/simple/packaging/
Downloading https://files.pythonhosted.org/packages/3e/89/7ea760b4daa42653ece2380531c90f64788d979110a2ab51049d92f408af/packaging-20.9-py2.py3-none-any.whl#sha256=67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21f051443c20947a
Best match: packaging 20.9
Processing packaging-20.9-py2.py3-none-any.whl
Installing packaging-20.9-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/packaging-20.9-py3.8.egg/EGG-INFO/requires.txt
Adding packaging 20.9 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/packaging-20.9-py3.8.egg
Searching for googleapis-common-protos<2.0dev,>=1.6.0
Reading https://pypi.org/simple/googleapis-common-protos/
Downloading https://files.pythonhosted.org/packages/b4/84/1285880ebf18f91fdba9d11a0385f6c4b48f5b20459f84d4ee7fd3415ebc/googleapis_common_protos-1.53.0.dev2-py2.py3-none-any.whl#sha256=ef07666b75aa9f02e5bf8ea1d9c852f19df1f8d9aa4fa16d32f7ac965b66734e
Best match: googleapis-common-protos 1.53.0.dev2
Processing googleapis_common_protos-1.53.0.dev2-py2.py3-none-any.whl
Installing googleapis_common_protos-1.53.0.dev2-py2.py3-none-any.whl to /usr/local/lib/python3.8/site-packages
writing requirements to /usr/local/lib/python3.8/site-packages/googleapis_common_protos-1.53.0.dev2-py3.8.egg/EGG-INFO/requires.txt
Adding googleapis-common-protos 1.53.0.dev2 to easy-install.pth file

Installed /usr/local/lib/python3.8/site-packages/googleapis_common_protos-1.53.0.dev2-py3.8.egg
error: google-auth 2.0.0.dev0 is installed but google-auth<2.0dev,>=1.21.1 is required by {'google-api-core'}
```
</details>
  

  5. If you're paranoid about dependencies being left behind, run `pip3 freeze | xargs pip3 uninstall -y` and verify only `pip`, `wheel`, and `setuptools` are installed before continuing.
  6. Build `FooModule` as a wheel: `python3 setup.py bdist_wheel`
  7. Try to install the wheel: `pip3 install dist/FooModule-0.0.1-py3-none-any.whl` This will resolve the dependencies fine and succeed!

#### What's going on?

This is my best guess. This package's dependencies are written as follows: https://github.com/googleapis/google-api-python-client/blob/91b61d3272de9b5aebad0cf1eb76ca53c24f22f9/setup.py#L34-L41

Notably, `google-api-core` itself has a [dependency](https://github.com/googleapis/python-api-core/blob/7273090a011c8b840d2e90e29dda3fc8f0eed792/setup.py#L34) on `google-auth`, `"google-auth >= 1.21.1, < 2.0dev"`.

Naively, these dependencies specify incompatible ranges. If `google-auth>=1.16.0` is installed first, then the resulting `google-auth` dependency in `google-api-core` will fail to be satisfied (`google-auth 2.0.0.dev0 is installed but google-auth<2.0dev,>=1.21.1 is required by {'google-api-core'}`).

However, it seems like pip and/or the wheel format are able to figure this out, and install a compatible dependency. easy_install and/or the egg format are unable to figure this out, and an error occurs.

Is this something that the google-api-python-client team is interested in fixing? I think the fix would be as simple as removing the dependency on `google-auth`, _or_ upper-bounding the dependency range to match `google-api-core`'s dependency range.

#### Workarounds in the meantime

1. Add an extra dependency `google-auth<2.0dev` to `FooModule`. This is what we're doing today. It might rely on the ordering of `install_requires`, which isn't guaranteed to be the installation order.
2. Build `FooModule` as a wheel. This is the more mature solution. We hope to do this eventually.



Discussion:
Hi @bbatliner,

> Is this something that the google-api-python-client team is interested in fixing? I think the fix would be as simple as removing the dependency on google-auth, or upper-bounding the dependency range to match google-api-core's dependency range.

I didn't realize there wasn't an upper bound on google-auth on this library, and I would be fine with adding it for consistency.

I would still recommend moving to a wheel though, since both `easy_install` and `bdist_egg` are deprecated. https://packaging.python.org/guides/distributing-packages-using-setuptools/#packaging-your-project walks through the current recommendations for packaging projects. Is creating/using a  source dist `python3 setup.py sdist` something that's feasible for your project if you're not able to build a wheel?

* easy_install deprecation: https://setuptools.readthedocs.io/en/latest/deprecated/easy_install.html 
* bdist_egg deprecation: https://setuptools.readthedocs.io/en/latest/userguide/commands.html#bdist-egg-create-a-python-egg-for-the-project


Thanks for the prompt reply! I opened #1180 to bound the dependency range.

I concur moving to wheel is best and we're aware of the deprecation. Thanks for the info.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -33,7 +33,7 @@
 
 install_requires = [
     "httplib2>=0.15.0,<1dev",
-    "google-auth>=1.16.0",
+    "google-auth>=1.16.0,<2dev",
     "google-auth-httplib2>=0.0.3",
     "google-api-core>=1.21.0,<2dev",
     "six>=1.13.0,<2dev",


Test patch: