Repository: GoogleCloudPlatform/professional-services

Issue:
Stacktrace with gcp-org-hierarchy-viewer (gcpohv)
Hello,

I was looking for a way to get a quick view of my folders and projects hierarchy and found this: https://github.com/GoogleCloudPlatform/professional-services/tree/main/tools/gcp-org-hierarchy-viewer

Unfortunately, this seems to be in a broken state / does not work with the instructions provided by the README.md. I suspect that since the code provided here is ~16 months old, it is no longer compatible with the current API client version.

When I try to simply run the command against my organization ID, I get the following stacktrace:

```
$ gcpohv -o <ID>
Traceback (most recent call last):
  File "/home/user/.local/bin/gcpohv", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/bin/gcpohv_cli.py", line 269, in main
    cli = Cli(args.key_file, args.organization, args.folder, args.use_id)
  File "/home/user/.local/bin/gcpohv_cli.py", line 101, in __init__
    self._crm_v1, self._crm_v2 = setup_clients(credentials_file)
  File "/home/user/.local/bin/gcpohv_cli.py", line 62, in setup_clients
    crm_v1 = discovery.build('cloudresourcemanager', 'v1', credentials=creds)
  File "/home/user/.local/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/googleapiclient/discovery.py", line 273, in build
    content = _retrieve_discovery_doc(
  File "/home/user/.local/lib/python3.8/site-packages/googleapiclient/discovery.py", line 387, in _retrieve_discovery_doc
    raise UnknownApiNameOrVersion("name: %s  version: %s" % (serviceName, version))
googleapiclient.errors.UnknownApiNameOrVersion: name: cloudresourcemanager  version: v1
```

I don't have the bandwidth to do any further investigation into, or to fix, this issue at the moment. However, my suspicion -- without digging deeper -- is that the `cloudresourcemanager v1` API is deprecated, and that the related code should be removed or wrapped in `try` statements to catch the error and move on without failing. For example here: https://github.com/GoogleCloudPlatform/professional-services/blob/bfc956a2ee8e0b6a0196bf60abfd631198a104d0/tools/gcp-org-hierarchy-viewer/gcpohv_cli.py#L62

System information:
```
$ uname -a
Linux 5.8.0-44-generic #50~20.04.1-Ubuntu SMP Wed Feb 10 21:07:30 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/issue.net
Linux Mint 20.1 Ulyssa

$ python --version
Python 3.8.5

$ pip list | grep google
google-api-core          1.26.0
google-api-python-client 2.0.0
google-auth              1.27.0
google-auth-httplib2     0.0.4
googleapis-common-protos 1.53.0
```

Stacktrace with gcp-org-hierarchy-viewer (gcpohv)
Hello,

I was looking for a way to get a quick view of my folders and projects hierarchy and found this: https://github.com/GoogleCloudPlatform/professional-services/tree/main/tools/gcp-org-hierarchy-viewer

Unfortunately, this seems to be in a broken state / does not work with the instructions provided by the README.md. I suspect that since the code provided here is ~16 months old, it is no longer compatible with the current API client version.

When I try to simply run the command against my organization ID, I get the following stacktrace:

```
$ gcpohv -o <ID>
Traceback (most recent call last):
  File "/home/user/.local/bin/gcpohv", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/bin/gcpohv_cli.py", line 269, in main
    cli = Cli(args.key_file, args.organization, args.folder, args.use_id)
  File "/home/user/.local/bin/gcpohv_cli.py", line 101, in __init__
    self._crm_v1, self._crm_v2 = setup_clients(credentials_file)
  File "/home/user/.local/bin/gcpohv_cli.py", line 62, in setup_clients
    crm_v1 = discovery.build('cloudresourcemanager', 'v1', credentials=creds)
  File "/home/user/.local/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/googleapiclient/discovery.py", line 273, in build
    content = _retrieve_discovery_doc(
  File "/home/user/.local/lib/python3.8/site-packages/googleapiclient/discovery.py", line 387, in _retrieve_discovery_doc
    raise UnknownApiNameOrVersion("name: %s  version: %s" % (serviceName, version))
googleapiclient.errors.UnknownApiNameOrVersion: name: cloudresourcemanager  version: v1
```

I don't have the bandwidth to do any further investigation into, or to fix, this issue at the moment. However, my suspicion -- without digging deeper -- is that the `cloudresourcemanager v1` API is deprecated, and that the related code should be removed or wrapped in `try` statements to catch the error and move on without failing. For example here: https://github.com/GoogleCloudPlatform/professional-services/blob/bfc956a2ee8e0b6a0196bf60abfd631198a104d0/tools/gcp-org-hierarchy-viewer/gcpohv_cli.py#L62

System information:
```
$ uname -a
Linux 5.8.0-44-generic #50~20.04.1-Ubuntu SMP Wed Feb 10 21:07:30 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/issue.net
Linux Mint 20.1 Ulyssa

$ python --version
Python 3.8.5

$ pip list | grep google
google-api-core          1.26.0
google-api-python-client 2.0.0
google-auth              1.27.0
google-auth-httplib2     0.0.4
googleapis-common-protos 1.53.0
```



Discussion:
@lhriley There might be something more complex going on, but I'll have to dig in a bit more... I'm on a Mac in a fresh and empty virtual environment, using `pyenv`, and it's installing and working for me.

```
✔ ~
09:54 $ pyenv versions
  system
* 3.8.7 (set by /Users/durivage/.pyenv/version)

✔ ~
09:54 $ mktmpenv
created virtual environment CPython3.8.7.final.0-64 in 435ms
  creator CPython3Posix(dest=/Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/durivage/Library/Application Support/virtualenv)
    added seed packages: pip==21.0.1, setuptools==53.0.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
virtualenvwrapper.user_scripts creating /Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/get_env_details
This is a temporary environment. It will be deleted when you run 'deactivate'.

(tmp-67d48790ccf8cc3) ✔ ~/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3
09:54 $ pip3 install 'git+https://github.com/GoogleCloudPlatform/professional-services.git#egg=gcp-org-hierarchy-viewer&subdirectory=tools/gcp-org-hierarchy-viewer'
Collecting gcp-org-hierarchy-viewer
  Cloning https://github.com/GoogleCloudPlatform/professional-services.git to /private/var/folders/2z/482cy6696qlcdkghq4r8mbrh00kpks/T/pip-install-oabxclu_/gcp-org-hierarchy-viewer_da0628c4bc664ab281ccdb67887110f8
  Running command git clone -q https://github.com/GoogleCloudPlatform/professional-services.git /private/var/folders/2z/482cy6696qlcdkghq4r8mbrh00kpks/T/pip-install-oabxclu_/gcp-org-hierarchy-viewer_da0628c4bc664ab281ccdb67887110f8
Collecting google-api-python-client>=1.7.8
  Downloading google_api_python_client-2.1.0-py2.py3-none-any.whl (6.6 MB)
     |████████████████████████████████| 6.6 MB 7.0 MB/s
Collecting asciitree==0.3.3
... (truncated intentionally)
Successfully built gcp-org-hierarchy-viewer
Installing collected packages: six, pyasn1, urllib3, rsa, pyparsing, pyasn1-modules, protobuf, idna, chardet, certifi, cachetools, requests, pytz, packaging, httplib2, googleapis-common-protos, google-auth, uritemplate, google-auth-httplib2, google-api-core, google-api-python-client, asciitree, gcp-org-hierarchy-viewer
Successfully installed asciitree-0.3.3 cachetools-4.2.1 certifi-2020.12.5 chardet-4.0.0 gcp-org-hierarchy-viewer-1.0 google-api-core-1.26.3 google-api-python-client-2.1.0 google-auth-1.28.0 google-auth-httplib2-0.1.0 googleapis-common-protos-1.53.0 httplib2-0.19.1 idna-2.10 packaging-20.9 protobuf-3.15.7 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyparsing-2.4.7 pytz-2021.1 requests-2.25.1 rsa-4.7.2 six-1.15.0 uritemplate-3.0.1 urllib3-1.26.4

(tmp-67d48790ccf8cc3) ✔ ~/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3
09:54 $ which gcpohv
/Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/gcpohv
(tmp-67d48790ccf8cc3) ✔ ~/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3

09:54 $ gcpohv -o hamsandwich.es
... (truncated intentionally)
```

@angstwad hmm. That is very strange. I just tested again with a fresh `virtualenv` and using your log as a guide. The result was the exact same issue.

Just to be sure this wasn't an API configuration issue, I verified that the `cloudresourcemanager.googleapis.com` API is enabled and refreshed my `gcloud` credentials. Are there any other APIs that are expected to be enabled, but are undocumented?
No, just the v1 and v2 resourcemanager APIs [here](https://github.com/GoogleCloudPlatform/professional-services/blob/main/tools/gcp-org-hierarchy-viewer/gcpohv_cli.py#L62-L63).  Something silly is going on.  I'll have to dig in.
Thanks @angstwad 
@lhriley If you do a `curl -i https://cloudresourcemanager.googleapis.com/$discovery/rest?version=v1`, what do you get? Do you have a proxy maybe?
@lhriley There might be something more complex going on, but I'll have to dig in a bit more... I'm on a Mac in a fresh and empty virtual environment, using `pyenv`, and it's installing and working for me.

```
✔ ~
09:54 $ pyenv versions
  system
* 3.8.7 (set by /Users/durivage/.pyenv/version)

✔ ~
09:54 $ mktmpenv
created virtual environment CPython3.8.7.final.0-64 in 435ms
  creator CPython3Posix(dest=/Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/durivage/Library/Application Support/virtualenv)
    added seed packages: pip==21.0.1, setuptools==53.0.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
virtualenvwrapper.user_scripts creating /Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/get_env_details
This is a temporary environment. It will be deleted when you run 'deactivate'.

(tmp-67d48790ccf8cc3) ✔ ~/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3
09:54 $ pip3 install 'git+https://github.com/GoogleCloudPlatform/professional-services.git#egg=gcp-org-hierarchy-viewer&subdirectory=tools/gcp-org-hierarchy-viewer'
Collecting gcp-org-hierarchy-viewer
  Cloning https://github.com/GoogleCloudPlatform/professional-services.git to /private/var/folders/2z/482cy6696qlcdkghq4r8mbrh00kpks/T/pip-install-oabxclu_/gcp-org-hierarchy-viewer_da0628c4bc664ab281ccdb67887110f8
  Running command git clone -q https://github.com/GoogleCloudPlatform/professional-services.git /private/var/folders/2z/482cy6696qlcdkghq4r8mbrh00kpks/T/pip-install-oabxclu_/gcp-org-hierarchy-viewer_da0628c4bc664ab281ccdb67887110f8
Collecting google-api-python-client>=1.7.8
  Downloading google_api_python_client-2.1.0-py2.py3-none-any.whl (6.6 MB)
     |████████████████████████████████| 6.6 MB 7.0 MB/s
Collecting asciitree==0.3.3
... (truncated intentionally)
Successfully built gcp-org-hierarchy-viewer
Installing collected packages: six, pyasn1, urllib3, rsa, pyparsing, pyasn1-modules, protobuf, idna, chardet, certifi, cachetools, requests, pytz, packaging, httplib2, googleapis-common-protos, google-auth, uritemplate, google-auth-httplib2, google-api-core, google-api-python-client, asciitree, gcp-org-hierarchy-viewer
Successfully installed asciitree-0.3.3 cachetools-4.2.1 certifi-2020.12.5 chardet-4.0.0 gcp-org-hierarchy-viewer-1.0 google-api-core-1.26.3 google-api-python-client-2.1.0 google-auth-1.28.0 google-auth-httplib2-0.1.0 googleapis-common-protos-1.53.0 httplib2-0.19.1 idna-2.10 packaging-20.9 protobuf-3.15.7 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyparsing-2.4.7 pytz-2021.1 requests-2.25.1 rsa-4.7.2 six-1.15.0 uritemplate-3.0.1 urllib3-1.26.4

(tmp-67d48790ccf8cc3) ✔ ~/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3
09:54 $ which gcpohv
/Users/durivage/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3/bin/gcpohv
(tmp-67d48790ccf8cc3) ✔ ~/Projects/Python/virtualenvs/tmp-67d48790ccf8cc3

09:54 $ gcpohv -o hamsandwich.es
... (truncated intentionally)
```

@angstwad hmm. That is very strange. I just tested again with a fresh `virtualenv` and using your log as a guide. The result was the exact same issue.

Just to be sure this wasn't an API configuration issue, I verified that the `cloudresourcemanager.googleapis.com` API is enabled and refreshed my `gcloud` credentials. Are there any other APIs that are expected to be enabled, but are undocumented?
No, just the v1 and v2 resourcemanager APIs [here](https://github.com/GoogleCloudPlatform/professional-services/blob/main/tools/gcp-org-hierarchy-viewer/gcpohv_cli.py#L62-L63).  Something silly is going on.  I'll have to dig in.
Thanks @angstwad 
@lhriley If you do a `curl -i https://cloudresourcemanager.googleapis.com/$discovery/rest?version=v1`, what do you get? Do you have a proxy maybe?

Resolving patch:
diff --git a/tools/gcp-org-hierarchy-viewer/gcpohv_cli.py b/tools/gcp-org-hierarchy-viewer/gcpohv_cli.py
--- a/tools/gcp-org-hierarchy-viewer/gcpohv_cli.py
+++ b/tools/gcp-org-hierarchy-viewer/gcpohv_cli.py
@@ -59,8 +59,14 @@ def setup_clients(credentials_file):
           Manager.
     """
     creds = setup_credentials(credentials_file)
-    crm_v1 = discovery.build('cloudresourcemanager', 'v1', credentials=creds)
-    crm_v2 = discovery.build('cloudresourcemanager', 'v2', credentials=creds)
+    crm_v1 = discovery.build('cloudresourcemanager',
+                             'v1',
+                             credentials=creds,
+                             static_discovery=True)
+    crm_v2 = discovery.build('cloudresourcemanager',
+                             'v2',
+                             credentials=creds,
+                             static_discovery=True)
     return crm_v1, crm_v2
 
 
@@ -85,7 +91,6 @@ def resource_filter(value, resource, field='displayName'):
 
 class Cli:
     """ Represents the entire CLI, UX, and the logic required to support it. """
-
     def __init__(self, credentials_file, organization, folder, use_org_id):
         """ Inits Cli class.
 
@@ -112,16 +117,16 @@ def _get_projects(self):
         fold_parents = collections.defaultdict(list)
 
         gen = (p for p in projs['projects']
-               if 'parent' in p and p['parent']['type'] == 'folder' and
-               p['lifecycleState'] == 'ACTIVE')
+               if 'parent' in p and p['parent']['type'] == 'folder'
+               and p['lifecycleState'] == 'ACTIVE')
         for p in gen:
             fold_parents[p['parent']['id']].append(p)
 
         org_parents = collections.defaultdict(list)
 
         gen = (p for p in projs['projects']
-               if 'parent' in p and p['parent']['type'] == 'organization' and
-               p['lifecycleState'] == 'ACTIVE')
+               if 'parent' in p and p['parent']['type'] == 'organization'
+               and p['lifecycleState'] == 'ACTIVE')
         for p in gen:
             org_parents[p['parent']['id']].append(p)
 
@@ -175,10 +180,9 @@ def recurse_and_attach(parent, tree):
                 raise SystemExit('No organizations found')
 
             org_id = [
-                org['name'].split('/')[1]
-                for org in orgs
-                if self.organization in org['displayName'] or
-                self.organization in org['name']
+                org['name'].split('/')[1] for org in orgs
+                if self.organization in org['displayName']
+                or self.organization in org['name']
             ][0]
 
             if self.use_org_id:
diff --git a/tools/gcp-org-hierarchy-viewer/setup.py b/tools/gcp-org-hierarchy-viewer/setup.py
--- a/tools/gcp-org-hierarchy-viewer/setup.py
+++ b/tools/gcp-org-hierarchy-viewer/setup.py
@@ -24,7 +24,7 @@
       scripts=['gcpohv_cli.py'],
       entry_points={'console_scripts': ['gcpohv=gcpohv_cli:main']},
       install_requires=[
-          'google-api-python-client>=1.7.8',
+          'google-api-python-client>=2.15.0',
           'asciitree==0.3.3',
       ],
       classifiers=[


Test patch: