Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
Relax validation of hub.db.type
### Proposed change

With the newly added schema validation (thanks for that, btw!), it's no longer possible to specify a custom database type (outside of the predefined set). It might be a good idea to relax schema validation for `hub.db.type`.

In our environment, we decided to stick with CocroachDB instead of PostgreSQL due to easier clusterization and scalability.
To make it work, we had to do the following:
* install sqlalchemy-cocroachdb before hub starts;
* fix a tiny defect in jupyterhub (https://github.com/jupyterhub/jupyterhub/pull/3484, it was not part of `1.4.1`).
So far, everything seems to work well.

The real question here is the following:
Should `hub.db.type` represent only the set of database drivers that are available out-of-the-box?

### Alternative options

As far as I know, in our case, specifying `postgres` is not going to have any negative effects. It only instructs `jupyterhub_config.py` to set an additional environment variable. So, we can live with that.

If you think that `hub.db.type` should stay as it is now, maybe adding a special comment like ("set it to `postgres` in case you decide to rely on an alternative sqlalchemy driver").

Personally, I'm fine with any of the options. Just having a clarification on what's the recommended approach would be great as well.

### Who would use this feature?

I saw somewhere in jupyterhub's docs that it should work with any sqlalchemy driver. So, I guess, I'm not the only one who relies on cocroachdb on something else.



Discussion:
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
<br>If you haven't done so already, check out [Jupyter's Code of Conduct](https://github.com/jupyter/governance/blob/master/conduct/code_of_conduct.md). Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
![welcome](https://raw.githubusercontent.com/jupyterhub/.github/master/images/welcome.jpg)
You can meet the other [Jovyans](https://jupyter.readthedocs.io/en/latest/community/content-community.html?highlight=jovyan#what-is-a-jovyan) by joining our [Discourse forum](http://discourse.jupyter.org/). There is also an intro thread there where you can stop by and say Hi! :wave:
<br>Welcome to the Jupyter community! :tada:

> Should hub.db.type represent only the set of database drivers that are available out-of-the-box?

I'm inclined to think we should support `other` as another `hub.db.type` or similar, but not try to provide adjustments for all various providers out there. Using `other` would imply that `hub.db.url` would be used, but that the password would be provided either directly via the URL or by manually mounting a environment variable or similar.

What do you think @weisdd?

### Relevant references to `hub.db.type`

https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/95982ad397136ff90e4e8830f6814d63599e31f1/jupyterhub/files/hub/jupyterhub_config.py#L61-L74

https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/95982ad397136ff90e4e8830f6814d63599e31f1/jupyterhub/schema.yaml#L743-L748


I'm somewhat open to supporting other DB providers by installing some python package as well, and setting some password in an env var like we do for `hub.db.url` as well if someone champions it. I just don't want there to be for example 10+ additional providers added.
@consideRatio If we expand a number of supported DB providers, that will set an implicit expectation that jupyterhub is thoroughly tested with all of them, which is not the case (I guess it requires adjustments to tests in CI).
On the other hand, if we just add the "other" option to `values.schema.json`, it'll sound like "use it at your own risk", which is a fair statement.

So, I think, `other` is a preferable option for now.
And if, at a certain point, we feel comfortable expanding the list of officially supported DB providers, then great, we can do that as well :)
@consideRatio As for installing packages for additional DB drivers:

Currently, I install it like this:

```yaml
hub:
  command:
    - "/usr/bin/bash"
    - "-ec"
    - set -exo pipefail; pip3 install -Iv sqlalchemy-cockroachdb==1.3.3; exec jupyterhub "$@"
```

Of course, it'd be great to have it pre-installed. Would it be fine if I open a PR with exactly that change (just adding the driver to the requirements) or there's a bigger list of things to do before I can expect such a change to be potentially approved?
I don't want to be the sole approver of adding that package but I'm positively inclined for that change as I see it doesn't seem to depend on lots of other python libraries etc in it.

You can go ahead and open a PR about that. It should be added to requirements.in and then the images/hub/dependencies script should run as well, hmm, which will update lots of dependencies I think - but yes that is what should be done I think.

---

@weisdd what does the `-e` part following `/usr/bin/bash` do? I don't see it listed in `man bash` output.
@consideRatio `-e` is documented here: https://www.gnu.org/software/bash/manual/bash.html#Modifying-Shell-Behavior
Basically, it makes the whole script fail if any of the commands within the pipeline exit with non-zero status. For example:

**with -e**:

```bash
cat << EOF > 2.sh
#!/usr/bin/bash
set -exo pipefail
sdfsdfsdf
echo "test"
EOF
chmod +x 2.sh
```

```shell
./2.sh 
+ sdfsdfsdf
./2.sh: line 3: sdfsdfsdf: command not found
```

**without -e**:

```bash
cat << EOF > 2.sh
#!/usr/bin/bash
set -xo pipefail
sdfsdfsdf
echo "test"
EOF
chmod +x 2.sh
```

```shell
./2.sh
+ sdfsdfsdf
./2.sh: line 3: sdfsdfsdf: command not found
+ echo test
test
```

As you can see, in the second example, the script continues to run despite the failure along the way (missing "sdfsdfsdf" binary).

So, basically, I want jupyterhub container to fail to start in case pip cannot install the DB driver. Of course, jupyterhub will eventually fail anyway in that case (sqlalchemy verifies urls against the list of installed drivers), so `-e` is redundant. It's more like a habbit to add it everywhere to make scripts fail early :) I even added it here twice :D (in `-ec` and in `set -exo`) - As long as options are set to the same values, it's fine :)

Resolving patch:
diff --git a/jupyterhub/files/hub/jupyterhub_config.py b/jupyterhub/files/hub/jupyterhub_config.py
--- a/jupyterhub/files/hub/jupyterhub_config.py
+++ b/jupyterhub/files/hub/jupyterhub_config.py
@@ -72,6 +72,8 @@ def camelCaseify(s):
         os.environ["MYSQL_PWD"] = db_password
     elif db_type == "postgres":
         os.environ["PGPASSWORD"] = db_password
+    else:
+        print(f"Warning: hub.db.password is ignored for hub.db.type={db_type}")
 
 
 # c.JupyterHub configuration from Helm chart's configmap


Test patch: