Repository: conbench/conbench (language: python)

Issue:
explore: require GITHUB_API_TOKEN for local dev?
This relates to https://github.com/conbench/conbench/issues/502 and #512.

I think we tend to _require_  GITHUB_API_TOKEN _even_ for local development purposes.

If we really require this, I think it's a good idea to document how to create such a token, so that we can instruct people to create one with limited scope (exclusively read permissions, for example).




Discussion / hints:

Fix patch:
diff --git a/.github/workflows/actions.yml b/.github/workflows/actions.yml
index 7e4436acb..fc5cf0621 100755
--- a/.github/workflows/actions.yml
+++ b/.github/workflows/actions.yml
@@ -8,6 +8,7 @@ on:
 
 env:
   PYTHON_VERSION: "3.8"
+  GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
 jobs:
   build:
@@ -76,4 +77,3 @@ jobs:
             make db-populate && make db-populate
           make teardown-app
           make test-run-app-dev
-
diff --git a/README.md b/README.md
index b0c6f1425..2bb137b2a 100755
--- a/README.md
+++ b/README.md
@@ -69,6 +69,8 @@ These need to be set up in your environment.
 
 The following `make` commands assume to be run in the root folder of a local repository clone.
 
+Before using these commands, set up a GitHub API token using [GitHub's instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token). It's recommended to only give the token read-only permissions to public repositories (which is the default for fine-grained personal access tokens). Set the `GITHUB_API_TOKEN` environment variable to that token value. If you do not have the token set, certain behaviors may not function correctly.
+
 #### Start application
 
 `make run-app` is a simple way to start and experiment with Conbench locally.
diff --git a/docker-compose.yml b/docker-compose.yml
index a70cee467..02b53c1db 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -46,6 +46,7 @@ services:
       GOOGLE_CLIENT_SECRET: AnotherStaticSecret
       CONBENCH_OIDC_ISSUER_URL: http://dex:5556/dex-for-conbench
       CONBENCH_INTENDED_BASE_URL: http://127.0.0.1:5000/
+      GITHUB_API_TOKEN: $GITHUB_API_TOKEN
     healthcheck:
       test: "curl -sfS http://0.0.0.0:5000/api/ping/"
       interval: 5s


Test patch:
diff --git a/conbench/tests/entities/test_commit.py b/conbench/tests/entities/test_commit.py
index 3c0e32691..82c2b8b69 100644
--- a/conbench/tests/entities/test_commit.py
+++ b/conbench/tests/entities/test_commit.py
@@ -86,10 +86,8 @@ def test_get_github_commit_none():
 )
 def test_get_github_commit_and_fork_point_sha(branch):
     # NOTE: This integration test intentionally hits GitHub.
-    if os.getenv("BUILDKITE"):
-        pytest.skip(
-            "Buildkite doesn't have a GITHUB_API_TOKEN so we won't hit the GitHub API"
-        )
+    if not os.getenv("GITHUB_API_TOKEN"):
+        pytest.skip("No GITHUB_API_TOKEN given so we won't hit the GitHub API")
 
     repo = "https://github.com/apache/arrow"
     sha = "3decc46119d583df56c7c66c77cf2803441c4458"
@@ -121,10 +119,8 @@ def test_get_github_commit_and_fork_point_sha(branch):
 )
 def test_get_github_commit_and_fork_point_sha_pull_request(branch, pr_number):
     # NOTE: This integration test intentionally hits GitHub.
-    if os.getenv("BUILDKITE"):
-        pytest.skip(
-            "Buildkite doesn't have a GITHUB_API_TOKEN so we won't hit the GitHub API"
-        )
+    if not os.getenv("GITHUB_API_TOKEN"):
+        pytest.skip("No GITHUB_API_TOKEN given so we won't hit the GitHub API")
 
     repo = "https://github.com/apache/arrow"
     sha = "982023150ccbb06a6f581f6797c017492485b58c"
@@ -146,10 +142,8 @@ def test_get_github_commit_and_fork_point_sha_pull_request(branch, pr_number):
 
 def test_backfill_default_branch_commits():
     # NOTE: This integration test intentionally hits GitHub.
-    if os.getenv("BUILDKITE"):
-        pytest.skip(
-            "Buildkite doesn't have a GITHUB_API_TOKEN so we won't hit the GitHub API"
-        )
+    if not os.getenv("GITHUB_API_TOKEN"):
+        pytest.skip("No GITHUB_API_TOKEN given so we won't hit the GitHub API")
 
     repository = "https://github.com/conbench/conbench"
     default_branch = "conbench:main"
diff --git a/conbench/tests/populate_local_conbench.py b/conbench/tests/populate_local_conbench.py
index 991a2a3ee..e8c2b0c8a 100644
--- a/conbench/tests/populate_local_conbench.py
+++ b/conbench/tests/populate_local_conbench.py
@@ -1,6 +1,3 @@
-# Setting a GitHub API token in the GITHUB_API_TOKEN env var will let this
-# population script find history + get commit details too
-
 import datetime
 import os
 import uuid