Repository: buildbot/buildbot

Issue:
Authentication error due to string/bytes confusion
Hi,

I use buildbot in a Docker swarm setup.

I use the buildbot-master image provided on dockerhub, in which I just add my config files and some ssh keys.
I use an NGinx container as a reverse proxy to provide HTTPS access and LDAP authentication. The buildbot-master configuration uses `util.RemoteUserAuth()` and the `REMOTEUSER` header.

This works great with the buildbot-master image v1.2.0

When v1.4.0 got out, I decided to give it a shot and noticed the image had switched to python3, nice!

I fixed my configuration files for Python3 and the server started as expected.

But when I tried to connect to the WebUI, I got this error:

```
 2018-09-04 10:01:09+0000 [-] Loading /var/lib/buildbot/buildbot.tac...
 2018-09-04 10:01:10+0000 [-] Loaded.
 2018-09-04 10:01:10+0000 [-] twistd 18.7.0 (/usr/bin/python3.6 3.6.5) starting up.
 2018-09-04 10:01:10+0000 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
 2018-09-04 10:01:10+0000 [-] Starting BuildMaster -- buildbot.version: 2018.09.03
 2018-09-04 10:01:10+0000 [-] Loading configuration from '/var/lib/buildbot/master.cfg'
 2018-09-04 10:01:10+0000 [-] Creating Prometheus metrics
 2018-09-04 10:01:10+0000 [-] Setting up database with URL 'postgresql+psycopg2://buildbot:xxxx@db/buildbot'

[...]

 2018-09-03 14:27:59+0000 [_GenericHTTPChannelProtocol,0,10.0.4.2] While rendering resource:
 	Traceback (most recent call last):
 	  File "/usr/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
 	    result = g.send(result)
 	  File "/usr/lib/python3.6/site-packages/buildbot/www/config.py", line 126, in renderIndex
 	    yield self.config['auth'].maybeAutoLogin(request)
 	  File "/usr/lib/python3.6/site-packages/twisted/internet/defer.py", line 1613, in unwindGenerator
 	    return _cancellableInlineCallbacks(gen)
 	  File "/usr/lib/python3.6/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
 	    _inlineCallbacks(None, g, status)
 	--- <exception caught here> ---
 	  File "/usr/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
 	    current.result = callback(current.result, *args, **kw)
 	  File "/usr/lib/python3.6/site-packages/buildbot/www/resource.py", line 91, in failHttpError
 	    f.trap(Error)
 	  File "/usr/lib/python3.6/site-packages/twisted/python/failure.py", line 439, in trap
 	    self.raiseException()
 	  File "/usr/lib/python3.6/site-packages/twisted/python/failure.py", line 467, in raiseException
 	    raise self.value.with_traceback(self.tb)
 	  File "/usr/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
 	    current.result = callback(current.result, *args, **kw)
 	  File "/usr/lib/python3.6/site-packages/buildbot/www/resource.py", line 84, in failHttpRedirect
 	    f.trap(Redirect)
 	  File "/usr/lib/python3.6/site-packages/twisted/python/failure.py", line 439, in trap
 	    self.raiseException()
 	  File "/usr/lib/python3.6/site-packages/twisted/python/failure.py", line 467, in raiseException
 	    raise self.value.with_traceback(self.tb)
 	  File "/usr/lib/python3.6/site-packages/buildbot/www/config.py", line 126, in renderIndex
 	    yield self.config['auth'].maybeAutoLogin(request)
 	  File "/usr/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
 	    result = g.send(result)
 	  File "/usr/lib/python3.6/site-packages/buildbot/www/auth.py", line 123, in maybeAutoLogin
 	    res = self.headerRegex.match(header)
 	builtins.TypeError: cannot use a bytes pattern on a string-like object
 ```

 It looks like the header provided by `request.getHeader()` is a string and the `headerRegex` has been converted to a bytes pattern in 734c7c52ae7b0c1e6f1de57e49fb56813b8d5b10 which leads to this error.

```diff
diff --git a/master/buildbot/www/auth.py b/master/buildbot/www/auth.py
index 86a88bd91..4c3cb6189 100644
--- a/master/buildbot/www/auth.py
+++ b/master/buildbot/www/auth.py
@@ -104,7 +104,7 @@ class NoAuth(AuthBase):
 
 class RemoteUserAuth(AuthBase):
     header = b"REMOTE_USER"
-    headerRegex = re.compile(br"(?P<username>[^ @]+)@(?P<realm>[^ @]+)")
+    headerRegex = re.compile(r"(?P<username>[^ @]+)@(?P<realm>[^ @]+)")
 
     def __init__(self, header=None, headerRegex=None, **kwargs):
         AuthBase.__init__(self, **kwargs)
@@ -113,7 +113,7 @@ class RemoteUserAuth(AuthBase):
         if header is not None:
             self.header = header
         if headerRegex is not None:
-            self.headerRegex = re.compile(unicode2bytes(headerRegex))
+            self.headerRegex = re.compile(headerRegex)
 
     @defer.inlineCallbacks
     def maybeAutoLogin(self, request):
```

This patch seems to solve the problem but I'm not sure it is entirely correct or complete.

Also, the Dockerfile included in the v1.4.0-py3 image is not the right one, it is for python2.


Discussion:
Hi,
The Dockerfile for py3 is very recent:
https://github.com/buildbot/buildbot/pull/4296
I built the docker image yesterday after that PR from @rodrigc .

It looks weird indeed that byte string is given to re.compile.
Can you submit a PR with this change?


Be careful.  Some of the auth functions in Twisted take `bytes`.
Sure, I'm looking how to do this properly.
in this case, the regex definitivly needs strings, so the headers have to be converted to strings, not the bytes.

Resolving patch:
diff --git a/master/buildbot/www/auth.py b/master/buildbot/www/auth.py
--- a/master/buildbot/www/auth.py
+++ b/master/buildbot/www/auth.py
@@ -111,7 +111,7 @@ def __init__(self, header=None, headerRegex=None, **kwargs):
         if self.userInfoProvider is None:
             self.userInfoProvider = UserInfoProviderBase()
         if header is not None:
-            self.header = header
+            self.header = unicode2bytes(header)
         if headerRegex is not None:
             self.headerRegex = re.compile(unicode2bytes(headerRegex))
 


Test patch: