Repository: google/clusterfuzz

Issue:
Local clusterfuzz instance >=v1.9.0 fails to install properly on Ubuntu 18.04
As requested by @inferno-chromium :)

## Faulty behavior
Local clusterfuzz instance >=v1.9.0 crashes when starting the server via `python butler.py run_server --bootstrap`

## Error
```
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started
```

## Related

#1328 

## Reproduction
Following the *'Prerequisites'* steps [here](https://google.github.io/clusterfuzz/getting-started/prerequisites/) with:

 * Host: stock Ubuntu 18.04
 * Clusterfuzz: `git clone https://github.com/google/clusterfuzz && git checkout tags/v1.9.0`
 * Python: 3.7 via `sudo add-apt-repository ppa:deadsnakes/ppa && sudo apt install -y python3.7`
 * Golang: v1.14.2, Install as described in [docs](https://golang.org/doc/install?download=go1.14.2.linux-amd64.tar.gz)

Next we run `local/install_deps.bash`. This runs in a couple of errors as shown below:


### Missing dependencies:

* curl (because Ubuntu...)
* pipenv (install_deps.bash tries to install it via *apt* but there is no such package on 18.04)
  * Workaround here is fix the installer (`local/install_deps_linux.bash`) using pip insead. E.g. replace the `apt install pipenv` with `sudo -EH pip3 install -U pipenv`

### Failed package installs

* psutil from Pipfile.lock fails to build/install due to missing `#include <Python.h>`
  * Fix here is to also install `libpython3.7` and `libpython3.7-dev`

When doing these fixes the installer finishes even though it still prints a bunch of errors regarding incompatible requirements:

```
| ERROR: grpcio-tools 1.17.0 has requirement grpcio>=1.17.0, but you'll have grpcio 1.15.0 which is incompatible.
| ERROR: astroid 2.3.3 has requirement wrapt==1.11.*, but you'll have wrapt 1.12.1 which is incompatible.
| ERROR: google-api-core 1.17.0 has requirement google-auth<2.0dev,>=1.14.0, but you'll have google-auth 1.8.1 which is incompatible.
| ERROR: google-cloud-datastore 1.7.0 has requirement google-cloud-core<0.29dev,>=0.28.0, but you'll have google-cloud-core 1.3.0 which is incompatible.
| ERROR: google-cloud-storage 1.13.2 has requirement google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 1.3.0 which is incompatible.
| ERROR: google-cloud-storage 1.13.2 has requirement google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 1.3.0 which is incompatible.
| ERROR: google-cloud-datastore 1.7.0 has requirement google-cloud-core<0.29dev,>=0.28.0, but you'll have google-cloud-core 1.3.0 which is incompatible.
| ERROR: google-api-core 1.17.0 has requirement google-auth<2.0dev,>=1.14.0, but you'll have google-auth 1.8.1 which is incompatible.
| ERROR: grpcio-tools 1.17.0 has requirement grpcio>=1.17.0, but you'll have grpcio 1.15.0 which is incompatible.
| ERROR: astroid 2.3.3 has requirement wrapt==1.11.*, but you'll have wrapt 1.12.1 which is incompatible.
```

### Server bootstrap

Once the dependency installer is done we can run:

```
pipenv shell
# python butler.py --help
python butler.py run_server --bootstrap
```

This crashes right at the end.

```
Running: python polymer_bundler.py (cwd='local')
| Building templates for App Engine...
| App Engine templates built successfully.
Created symlink: source: /home/toor/clusterfuzz/local/storage/local_gcs, target /home/toor/clusterfuzz/src/appengine/local_gcs.
Running: gunicorn -b :9000 main:app (cwd='src/appengine')
| [2020-05-14 21:56:30 +0200] [15318] [INFO] Starting gunicorn 20.0.4
| [2020-05-14 21:56:30 +0200] [15318] [INFO] Listening at: http://0.0.0.0:9000 (15318)
| [2020-05-14 21:56:30 +0200] [15318] [INFO] Using worker: sync
| [2020-05-14 21:56:30 +0200] [15321] [INFO] Booting worker with pid: 15321
| [2020-05-14 21:56:36 +0200] [15321] [ERROR] Exception in worker process
| Traceback (most recent call last):
|   File "/home/toor/.local/share/virtualenvs/clusterfuzz-2lmaEd3m/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
|     worker.init_process()
|   File "/home/toor/.local/share/virtualenvs/clusterfuzz-2lmaEd3m/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process
|     self.load_wsgi()
|   File "/home/toor/.local/share/virtualenvs/clusterfuzz-2lmaEd3m/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
|     self.wsgi = self.app.wsgi()
|   File "/home/toor/.local/share/virtualenvs/clusterfuzz-2lmaEd3m/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
|     self.callable = self.load()
|   File "/home/toor/.local/share/virtualenvs/clusterfuzz-2lmaEd3m/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
|     return self.load_wsgiapp()
|   File "/home/toor/.local/share/virtualenvs/clusterfuzz-2lmaEd3m/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
|     return util.import_app(self.app_uri)
|   File "/home/toor/.local/share/virtualenvs/clusterfuzz-2lmaEd3m/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app
|     mod = importlib.import_module(module)
|   File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
|     return _bootstrap._gcd_import(name[level:], package, level)
|   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
|   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
|   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
|   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
|   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
|   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
|   File "/home/toor/clusterfuzz/src/appengine/main.py", line 32, in <module>
|     firebase_admin.initialize_app()
|   File "third_party/firebase_admin/__init__.py", line 65, in initialize_app
|     credential = credentials.ApplicationDefault()
|   File "third_party/firebase_admin/credentials.py", line 133, in __init__
|     self._g_credential, self._project_id = google.auth.default(scopes=_scopes)
|   File "/home/toor/clusterfuzz/src/third_party/google/auth/_default.py", line 321, in default
|     raise exceptions.DefaultCredentialsError(_HELP_MESSAGE)
| google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started
| [2020-05-14 21:56:36 +0200] [15321] [INFO] Worker exiting (pid: 15321)
| [2020-05-14 21:56:36 +0200] [15318] [INFO] Shutting down: Master
| [2020-05-14 21:56:36 +0200] [15318] [INFO] Reason: Worker failed to boot.
| Return code is non-zero (3).
| Exit.
Bootstrapping datastore...
Running: python butler.py run setup --non-dry-run --local --config-dir=configs/test
| Creating config
| Creating fuzzer afl
| Creating fuzzer libFuzzer
| Creating fuzzer honggfuzz
| Creating fuzzer syzkaller
| Creating template afl
| Creating template engine_asan
| Creating template engine_msan
| Creating template engine_ubsan
| Creating template honggfuzz
| Creating template libfuzzer
| Creating template syzkaller
| Creating template prune
| Done

```

## Expected behavior

Local instance runs fine when following the steps in the docs as they explicitly state that there is no need for gauth.

## Scope
This is not host OS related but must have been introduced post `tag v1.8.0` as this one runs fine. However, tag v1.9.0+ shows the above behavior.




Discussion:

Resolving patch:
diff --git a/src/appengine/main.py b/src/appengine/main.py
--- a/src/appengine/main.py
+++ b/src/appengine/main.py
@@ -24,12 +24,14 @@
 if os.path.exists(config_modules_path):
   sys.path.append(config_modules_path)
 
-if os.environ.get('GAE_ENV'):
+gae_env = os.environ.get('GAE_ENV')
+if gae_env:
   import pkg_resources
   importlib.reload(pkg_resources)
 
-  import firebase_admin
-  firebase_admin.initialize_app()
+  if gae_env != 'dev':
+    import firebase_admin
+    firebase_admin.initialize_app()
 
 try:
   # Run any module initialization code.


Test patch: