Repository: buildbot/buildbot

Issue:
GitHubPullrequestPoller token secret gives builtins.TypeError:
When attempting the following in GitHubPullrequestPoller:
```
token=util.Secret("githubToken"),
```
I receive: builtins.TypeError: must be str, not Secret

and similarly via Interpolate:
```
token=util.Interpolate("%(secret:githubToken)s"),
```
builtins.TypeError: must be str, not Interpolate

Apologies if I'm doing this wrong but given:
https://docs.buildbot.net/latest/manual/secretsmanagement.html#as-argument-to-services
I had expected this to work. I got the impression that the secret manager was as yet not supported across the board and so if this is an example of an unsupported use then it seems like a nice one to have.

Thanks all for a great project.

Incidentally in both cases I can cast via str() and avoid this issue but then run into another issue which I will report separately.


Discussion:
Also, with pointers to a similar past fix, if this is a missing secrets supported area; I'm happy to have a go at a pr to address it. Though it may take me some time as I've only just started looking at buildbot and it's code :). But would be good to get more familiar with it.
Hello, 
I am facing the same issue, I don't think a fix has been made for it?
Following is a thorough description of the error encountered:

When using the `GitHubPullRequestPoller`, looks like it is not possible to use a secret to fill the `token` argument
```python
changes.GitHubPullrequestPoller(
    owner='me',
    repo='my_repo',
    branches=['dev', 'master'],
    pollInterval=60,
    # Here is what I'd like to use
    token=util.Secret('githubToken'),
    # also tried that, but didn't work either
    # token=util.Interpolate('%(secret:githubToken)s'),
)
```

Here is the error produced:
```python
2020-05-19 07:32:38+0000 [-] while starting BuildMaster
	Traceback (most recent call last):
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
	    result = g.send(result)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/util/service.py", line 216, in reconfigServiceWithSibling
	    **kwargs)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1613, in unwindGenerator
	    return _cancellableInlineCallbacks(gen)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
	    _inlineCallbacks(None, g, status)
	--- <exception caught here> ---
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/master.py", line 304, in startService
	    yield self.reconfigServiceWithBuildbotConfig(self.config)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/util/service.py", line 53, in reconfigServiceWithBuildbotConfig
	    yield svc.reconfigServiceWithBuildbotConfig(new_config)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/process/measured_service.py", line 33, in reconfigServiceWithBuildbotConfig
	    yield super(MeasuredBuildbotServiceManager, self).reconfigServiceWithBuildbotConfig(new_config)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/util/service.py", line 505, in reconfigServiceWithBuildbotConfig
	    yield defer.maybeDeferred(child.setServiceParent, self)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/util/service.py", line 66, in setServiceParent
	    yield self.parent.addService(self)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/util/service.py", line 317, in startService
	    yield super(ClusteredBuildbotService, self).startService()
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/util/service.py", line 227, in startService
	    yield self.configureService()
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/util/service.py", line 216, in reconfigServiceWithSibling
	    **kwargs)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
	    result = g.send(result)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/changes/github.py", line 115, in reconfigService
	    http_headers.update({'Authorization': 'token ' + token})
	exceptions.TypeError: cannot concatenate 'str' and 'Secret' objects
```

Casting as a string using `str(util.Secret('githubToken'))` will fix this error and Buildmaster can start, but it is then producing following error when polling github:
```python
2020-05-19 07:43:52+0000 [-] GitHubPullrequestPoller: polling GitHub repository SunPower/SENSI, branches: ['dev', 'master']
2020-05-19 07:43:52+0000 [HTTP11ClientProtocol (TLSMemoryBIOProtocol),client] while polling for changes
	Traceback (most recent call last):
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
	    result = g.send(result)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/changes/github.py", line 278, in poll
	    yield self._processChanges(result)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1613, in unwindGenerator
	    return _cancellableInlineCallbacks(gen)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
	    _inlineCallbacks(None, g, status)
	--- <exception caught here> ---
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/changes/github.py", line 278, in poll
	    yield self._processChanges(result)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
	    result = g.send(result)
	  File "/home/ubuntu/virtualenv/venv_bb_master/local/lib/python2.7/site-packages/buildbot/changes/github.py", line 206, in _processChanges
	    base_branch = pr['base']['ref']
	exceptions.TypeError: string indices must be integers
```

Thanks for your help! 
Bumping this issue as I just ran into it, too. Hopefully this will be fixed in 3.0!

Resolving patch:
diff --git a/master/buildbot/changes/github.py b/master/buildbot/changes/github.py
--- a/master/buildbot/changes/github.py
+++ b/master/buildbot/changes/github.py
@@ -106,6 +106,7 @@ def reconfigService(self,
 
         http_headers = {'User-Agent': 'Buildbot'}
         if token is not None:
+            token = yield self.renderSecrets(token)
             http_headers.update({'Authorization': 'token ' + token})
 
         self._http = yield httpclientservice.HTTPClientService.getService(


Test patch:
diff --git a/master/buildbot/test/unit/changes/test_github.py b/master/buildbot/test/unit/changes/test_github.py
--- a/master/buildbot/test/unit/changes/test_github.py
+++ b/master/buildbot/test/unit/changes/test_github.py
@@ -20,7 +20,11 @@
 
 from buildbot.changes.github import GitHubPullrequestPoller
 from buildbot.config import ConfigErrors
+from buildbot.process.properties import Properties
+from buildbot.process.properties import Secret
+from buildbot.secrets.manager import SecretManager
 from buildbot.test.fake import httpclientservice as fakehttpclientservice
+from buildbot.test.fake.secrets import FakeSecretStorage
 from buildbot.test.util import changesource
 from buildbot.test.util.misc import TestReactorMixin
 
@@ -177,8 +181,17 @@ class TestGitHubPullrequestPoller(changesource.ChangeSourceMixin,
     def setUp(self):
         self.setUpTestReactor()
         yield self.setUpChangeSource()
+
+        fake_storage_service = FakeSecretStorage()
+
+        secret_service = SecretManager()
+        secret_service.services = [fake_storage_service]
+        yield secret_service.setServiceParent(self.master)
+
         yield self.master.startService()
 
+        fake_storage_service.reconfigService(secretdict={"token": "1234"})
+
     @defer.inlineCallbacks
     def tearDown(self):
         yield self.master.stopService()
@@ -193,6 +206,9 @@ def newChangeSource(self,
         http_headers = {'User-Agent': 'Buildbot'}
         token = kwargs.get('token', None)
         if token:
+            p = Properties()
+            p.master = self.master
+            token = yield p.render(token)
             http_headers.update({'Authorization': 'token ' + token})
         self._http = yield fakehttpclientservice.HTTPClientService.getService(
             self.master, self, endpoint, headers=http_headers)
@@ -235,6 +251,16 @@ def test_custom_name(self):
     def test_SimplePR(self):
         yield self.newChangeSource(
             'defunkt', 'defunkt', token='1234', github_property_whitelist=["github.*"])
+        yield self.simple_pr()
+
+    @defer.inlineCallbacks
+    def test_secret_token(self):
+        yield self.newChangeSource(
+            'defunkt', 'defunkt', token=Secret('token'), github_property_whitelist=["github.*"])
+        yield self.simple_pr()
+
+    @defer.inlineCallbacks
+    def simple_pr(self):
         self._http.expect(
             method='get',
             ep='/repos/defunkt/defunkt/pulls',