Repository: sopel-irc/sopel

Issue:
Ability to specify full DB connection URL
### The problem

I'm running sopel on heroku. Heroku manages the database, and provides the connection info as an environment variable `$DATABASE_URL`. This URL has the user/pass/host all baked into it, ready to pass to sqlalchemy. But there's currently no way to tell sopel to just use this. Instead, you have to manually break it into pieces and hard-code each piece in the config file. This is (a) extra work, (b) insecure b/c you have to write down the password in plaintext, (c) liable to break whenever heroku's infrastructure changes.

### The solution

For now, I've hacked up my copy of sopel to use $DATABASE_URL if it exists, and otherwise fall back on the normal config options:

https://github.com/sopel-irc/sopel/compare/master...njsmith:njs-heroku-support

This is maybe a bit *too* simple to be a general solution, but it would be very nice if upstream sopel had something *like* this, so I don't have to maintain a fork :-).



Discussion:
Sopel 7 added support for passing config values via environment variables (see #1096). [The unstable docs](https://unstable.docs.sopel.chat/cli.html#supported-environment-variables) based on `master` do a better job of exposing and explaining this feature, which is unfortunately not documented in the 7.0.x series.

Presumably Heroku will always give you a URL, per the variable name. If you can—for example—set environment variables from a Python script before Sopel runs, running [`urllib.parse.urlparse()`](https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlparse) on the value of `$DATABASE_URL` should give you the components you need to then set `SOPEL_CORE_DB_TYPE`, `SOPEL_CORE_DB_USER`, `SOPEL_CORE_DB_PASS`, etc. for Sopel to use.

_(Yes, a **lot** of the work for 7.1 has been about improving documentation, when we're not fixing bugs. 😸)_
Ah, that is handy!

Perhaps sopel could also have a `db_url` config option, that if present overrides the `db_user`/`db_pass`/etc. options? That seems simpler than dissecting a URL just so sopel can put it back together :-)
> Perhaps sopel could also have a `db_url` config option, that if present overrides the `db_user`/`db_pass`/etc. options?

Maybe for 8.0, we can look into it. For the remaining life of 7.x, though, you'll have to use the above approach. :)
oh, do you have a rule against adding features in 7.x point releases?
No rule against new features in minor releases per se, but it's long past time to release 7.1, which is _this close_ to being done. Don't want to add more stuff to it, and there isn't going to be a 7.2.

Resolving patch:
diff --git a/sopel/config/core_section.py b/sopel/config/core_section.py
--- a/sopel/config/core_section.py
+++ b/sopel/config/core_section.py
@@ -373,6 +373,23 @@ class CoreSection(StaticSection):
 
     """
 
+    db_url = ValidatedAttribute('db_url')
+    """A raw database URL.
+
+    If this option is present, Sopel will ignore **all** other ``db_*``
+    settings and use this option's value only.
+
+    .. note::
+
+        Specifying this option via the ``SOPEL_CORE_DB_URL`` :ref:`environment
+        variable <Overriding individual settings>` may prove especially useful
+        in certain cloud environments, avoiding the need to split a database
+        URI provided by the platform at runtime into its components with a
+        startup script.
+
+    .. versionadded:: 8.0
+    """
+
     db_user = ValidatedAttribute('db_user')
     """The user for Sopel's database.
 
diff --git a/sopel/db.py b/sopel/db.py
--- a/sopel/db.py
+++ b/sopel/db.py
@@ -9,7 +9,7 @@
 import traceback
 
 from sqlalchemy import Column, create_engine, ForeignKey, Integer, String
-from sqlalchemy.engine.url import URL
+from sqlalchemy.engine.url import make_url, URL
 from sqlalchemy.exc import OperationalError, SQLAlchemyError
 from sqlalchemy.ext.declarative import declarative_base
 from sqlalchemy.orm import scoped_session, sessionmaker
@@ -112,12 +112,17 @@ class SopelDB(object):
     """
 
     def __init__(self, config):
-        # MySQL - mysql://username:password@localhost/db
-        # SQLite - sqlite:////home/sopel/.sopel/default.db
-        self.type = config.core.db_type
-
-        # Handle SQLite explicitly as a default
-        if self.type == 'sqlite':
+        if config.core.db_url is not None:
+            self.url = make_url(config.core.db_url)
+
+            # TODO: there's no way to get `config.core.db_type.choices`, but
+            # it would be nice to validate this type name somehow. Shouldn't
+            # affect anything, since the only thing it's ever used for is
+            # checking whether the configured database is 'sqlite'.
+            self.type = self.url.drivername.split('+', 1)[0]
+        elif config.core.db_type == 'sqlite':
+            self.type = 'sqlite'
+            drivername = config.core.db_driver or 'sqlite'
             path = config.core.db_filename
             if path is None:
                 path = os.path.join(config.core.homedir, config.basename + '.db')
@@ -132,10 +137,10 @@ def __init__(self, config):
                     'core.db_filename is valid'.format(os.path.dirname(path)),
                     path
                 )
-            self.filename = path
-            self.url = 'sqlite:///%s' % path
-        # Otherwise, handle all other database engines
+            self.url = make_url('sqlite:///' + path)
         else:
+            self.type = config.core.db_type
+
             query = {}
             if self.type == 'mysql':
                 drivername = config.core.db_driver or 'mysql'
@@ -153,11 +158,11 @@ def __init__(self, config):
             else:
                 raise Exception('Unknown db_type')
 
-            db_user = config.core.db_user
-            db_pass = config.core.db_pass
-            db_host = config.core.db_host
+            db_user = config.core.db_user  # Sometimes empty
+            db_pass = config.core.db_pass  # Sometimes empty
+            db_host = config.core.db_host  # Sometimes empty
             db_port = config.core.db_port  # Optional
-            db_name = config.core.db_name  # Optional, depending on DB
+            db_name = config.core.db_name  # Sometimes optional
 
             # Ensure we have all our variables defined
             if db_user is None or db_pass is None or db_host is None:


Test patch: