Repository: ckan/ckan

Issue:
Secrets consolidation
These are to my knowledge the secrets related config options needed to run CKAN 2.10 (please mention any I missed below):
| config name | `ckan generate config` default value | fallback value |
| --- | --- | --- |
| `beaker.session.secret` | Random token | None |
| Flask's `SECRET_KEY` | Not added | `beaker.session.secret` |
| `WTF_FORMS_CSRF_SECRET` | Random token | `SECRET_KEY` |
| `api_token.jwt.encode.secret` | `string:%(beaker.session.secret)s` | `beaker.session.secret` |
| `api_token.jwt.decode.secret` | `string:%(beaker.session.secret)s` | `beaker.session.secret` |

So for historic reasons, `beaker.session.secret` ends up being the default fallback for all of these:

```mermaid
graph TD;
    api_token.jwt.encode.secret-->beaker.session.secret;
    api_token.jwt.decode.secret-->beaker.session.secret;
    SECRET_KEY-->beaker.session.secret;
    WTF_FORMS_CSRF_SECRET-->SECRET_KEY;
```
More things to note:

* `beaker.session.secret` is the one added to the ini file and the one all options fall back to, but if you don't set it on the ini file, the error you get is:
```
  File "/home/adria/dev/pyenvs/ckan-py3/src/ckan/ckan/config/middleware/flask_app.py", line 176, in make_flask_stack                      
    raise RuntimeError(u'You must provide a value for the secret key'                                                                 
RuntimeError: You must provide a value for the secret key with the SECRET_KEY config option
```
* You can use `SECRET_KEY` in your ini file instead, but if you don't define `beaker.session.secret`, API calls will fail:
```
  File "/home/adria/dev/pyenvs/ckan-py3/src/ckan/ckan/lib/api_token.py", line 84, in decode
    _get_secret(encode=False),                                       
  File "/home/adria/dev/pyenvs/ckan-py3/src/ckan/ckan/lib/api_token.py", line 39, in _get_secret
    secret = u"string:" + config.get(_config_secret_fallback)

```

Looks like there is space to consolidate how these settings are managed and improve these inconsistencies.
My feeling is that we should try to fall back to either `SECRET_KEY` or a new `ckan.secret_key` and consolidate how all keys are set in the ini file and how their defaults are handled. And think of ways of clearly documenting them.



Discussion:
Decisions from the dev meeting:

* Use `SECRET_KEY` as the main setting: set it on the ini file with `ckan generate config`
* Make all other settings use the `SECRET_KEY` value by default in the ini file (`string:%(SECRET_KEY)s`)
* Make all other settings fall back to `SECRET_KEY` in the code in case they are not defined


Resolving patch:
diff --git a/ckan/config/middleware/flask_app.py b/ckan/config/middleware/flask_app.py
--- a/ckan/config/middleware/flask_app.py
+++ b/ckan/config/middleware/flask_app.py
@@ -139,13 +139,6 @@ def make_flask_stack(conf: Union[Config, CKANConfig]) -> CKANApp:
 
     # Do all the Flask-specific stuff before adding other middlewares
 
-    # Secret key needed for flask-debug-toolbar and sessions
-    if not app.config.get('SECRET_KEY'):
-        app.config['SECRET_KEY'] = config.get('beaker.session.secret')
-    if not app.config.get('SECRET_KEY'):
-        raise RuntimeError(u'You must provide a value for the secret key'
-                           ' with the SECRET_KEY config option')
-
     root_path = config.get('ckan.root_path')
     if debug:
         from flask_debugtoolbar import DebugToolbarExtension
diff --git a/ckan/lib/api_token.py b/ckan/lib/api_token.py
--- a/ckan/lib/api_token.py
+++ b/ckan/lib/api_token.py
@@ -19,7 +19,7 @@
 
 _config_encode_secret = u"api_token.jwt.encode.secret"
 _config_decode_secret = u"api_token.jwt.decode.secret"
-_config_secret_fallback = u"beaker.session.secret"
+_config_secret_fallback = u"SECRET_KEY"
 
 _config_algorithm = u"api_token.jwt.algorithm"
 


Test patch:
diff --git a/ckan/tests/config/test_environment.py b/ckan/tests/config/test_environment.py
--- a/ckan/tests/config/test_environment.py
+++ b/ckan/tests/config/test_environment.py
@@ -109,3 +109,22 @@ def test_config_from_envs_are_normalized(ckan_config):
     environment.update_config()
 
     assert ckan_config["smtp.starttls"] is False
+
+
+@pytest.mark.ckan_config("SECRET_KEY", "super_secret")
+@pytest.mark.ckan_config("beaker.session.secret", None)
+@pytest.mark.ckan_config("beaker.session.validate_key", None)
+@pytest.mark.ckan_config("WTF_CSRF_SECRET_KEY", None)
+def test_all_secrets_default_to_SECRET_KEY(ckan_config):
+
+    environment.update_config()
+
+    for key in [
+        "SECRET_KEY",
+        "beaker.session.secret",
+        "beaker.session.validate_key",
+        "WTF_CSRF_SECRET_KEY",
+    ]:
+        assert ckan_config[key] == "super_secret"
+
+    # Note: api_token.jwt.*.secret are tested in ckan/tests/lib/test_api_token.py
diff --git a/ckan/tests/config/test_middleware.py b/ckan/tests/config/test_middleware.py
--- a/ckan/tests/config/test_middleware.py
+++ b/ckan/tests/config/test_middleware.py
@@ -78,10 +78,10 @@ def test_secret_key_is_used_if_present(app):
     assert app.flask_app.config[u"SECRET_KEY"] == u"super_secret_stuff"
 
 
-@pytest.mark.ckan_config(u"SECRET_KEY", None)
-def test_beaker_secret_is_used_by_default(app):
+@pytest.mark.ckan_config(u"SECRET_KEY", "some_secret")
+def test_SECRET_KEY_is_used_by_default(app):
     assert (
-        app.flask_app.config[u"SECRET_KEY"] == config[u"beaker.session.secret"]
+        app.flask_app.config[u"SECRET_KEY"] == "some_secret"
     )
 
 
diff --git a/ckan/tests/lib/test_api_token.py b/ckan/tests/lib/test_api_token.py
new file mode 100644
--- /dev/null
+++ b/ckan/tests/lib/test_api_token.py
@@ -0,0 +1,11 @@
+import pytest
+
+from ckan.lib.api_token import _get_secret
+
+
+@pytest.mark.ckan_config("SECRET_KEY", "super_secret")
+@pytest.mark.ckan_config("api_token.jwt.encode.secret", None)
+@pytest.mark.ckan_config("api_token.jwt.decode.secret", None)
+def test_secrets_default_to_SECRET_KEY():
+    assert _get_secret(True) == "super_secret"  # Encode
+    assert _get_secret(False) == "super_secret"  # Decode