Repository: conbench/conbench (language: python)

Issue:
Swagger docs don't give option to poke at live server
I'm currently looking at the Swagger docs for a conbench instance and the only server dropdown choice is localhost
![image](https://user-images.githubusercontent.com/6223243/194589781-dea24743-961e-4ad9-9e6c-1475d1f45d53.png)

I accessed the swagger docs by clicking on 'API Docs' after logging in to conbench, so I'm already auth'd to the server. I'd expect to see the active server as the default option on the above dropdown

![image](https://user-images.githubusercontent.com/6223243/194590212-2380a958-fda6-4ad6-93bc-37d6830464d4.png)



Discussion / hints:
A public example of swagger docs: https://conbench.ursa.dev/api/docs/#/

Could adding `DB_HOST` and `DB_PORT` to https://github.com/conbench/conbench/blob/fdb30f2934481989e981c49837506c0622d6bb3a/conbench/api/_docs.py#L16 be enough?
Hello! Naive question: Is "poke at live server" safe, especially considering mutating API calls with http methods such as POST, DELETE? This would after all change the state in some database.

I suppose the current state (where only localhost:5000 is listed) is tailored for a development environment where a dev server runs locally and where mutating API calls do not need to be carefully crafted (can ignorantly be sent w/o fear of messing up state).

Interaction of the swagger docs with the live server should maybe be limited to read-only API calls? Happy to learn more about the context.

> I suppose the current state (where only localhost:5000 is listed) is tailored for a development environment where a dev server runs locally and where mutating API calls do not need to be carefully crafted (can ignorantly be sent w/o fear of messing up state).

When I was running this last, I actually had to change this even during local development to go from localhost to 127.0.0.1. So even in that case it would be nice if the swagger docs protected this.

> Hello! Naive question: Is "poke at live server" safe, especially considering mutating API calls with http methods such as POST, DELETE? This would after all change the state in some database.

> Interaction of the swagger docs with the live server should maybe be limited to read-only API calls? Happy to learn more about the context.

Yeah, we might want to disable trying it out (copying Swagger's terms) for post and put on production servers. It would be nice, if we could, to be able to still have the `GET`s or other non-state changing calls work. And ideally, be able to set this so that one could turn this on for a staging server (we don't (yet) have a staging server for arrow, but we do have one internally).
Adding the quick thought that this topic relates strongly to the concept of user privileges. I found in the docs:

> By default, conbench has open read access, so a user account is not required to view results or read from the API. 




Fix patch:
diff --git a/conbench/api/_docs.py b/conbench/api/_docs.py
index 39a0eea56..cdfe2ddbb 100644
--- a/conbench/api/_docs.py
+++ b/conbench/api/_docs.py
@@ -13,7 +13,7 @@
         apispec_webframeworks.flask.FlaskPlugin(),
         apispec.ext.marshmallow.MarshmallowPlugin(),
     ],
-    servers=[{"url": "http://localhost:5000/"}],
+    servers=[{"url": Config.INTENDED_BASE_URL}],
 )
 
 
diff --git a/conbench/config.py b/conbench/config.py
index de3108ec1..9052d345c 100644
--- a/conbench/config.py
+++ b/conbench/config.py
@@ -25,6 +25,30 @@ class Config:
     # to recalculate history.
     DISTRIBUTION_COMMITS = int(os.environ.get("DISTRIBUTION_COMMITS", 100))
 
+    # The base URL (scheme, DNS name, path prefix) that regular HTTP clients
+    # are expected to use for reaching the HTTP server exposing the app.
+    # Depends on the deployment and cannot generally be determined by the app
+    # itself (requires human input). The default value is tailored to the Flask
+    # development HTTP server defaults (non-TLS, binds on 127.0.0.1, port
+    # 5000). Can be adjusted via the `--host` and `--port` command line flags
+    # when invoking `flask run`. Three interesting scenarios:
+    # - local dev setup with default listen address: no action needed
+    # - local dev setup with custom listen address: user should set meaningful
+    #   value
+    # - prod/staging setup: user should set meaningful value such as for
+    #   example https://conbench.ursa.dev/
+    #
+    # Currently used for populating the 'Servers' dropdown in the
+    # Swagger/OpenAPI docs website.
+    INTENDED_BASE_URL = os.environ.get(
+        "CONBENCH_INTENDED_BASE_URL", "http://127.0.0.1:5000/"
+    )
+    # Require trailing slash towards tidy URL generation.
+    # Note: might want to catch bad input via e.g.
+    # https://validators.readthedocs.io/en/latest/#module-validators.url
+    if not INTENDED_BASE_URL.endswith("/"):
+        INTENDED_BASE_URL += "/"
+
 
 class TestConfig(Config):
     DB_NAME = os.environ.get("DB_NAME", f"{APPLICATION_NAME.lower()}_test")


Test patch:
diff --git a/conbench/tests/api/_expected_docs.py b/conbench/tests/api/_expected_docs.py
index c491faa2d..becec63a8 100644
--- a/conbench/tests/api/_expected_docs.py
+++ b/conbench/tests/api/_expected_docs.py
@@ -1824,7 +1824,7 @@
             },
         },
     },
-    "servers": [{"url": "http://localhost:5000/"}],
+    "servers": [{"url": "http://127.0.0.1:5000/"}],
     "tags": [
         {"name": "Authentication"},
         {"description": "List of endpoints", "name": "Index"},