Repository: buildbot/buildbot

Issue:
util.RemoteUserAuth raises builtins.TypeError
RemoteUserAuth (nginx http basic auth) fails with Type error, because the result of the regex match has values of type `bytes`, e.g.: `{'username': b'foobar'}`
Full stacktrace
```python
        Traceback (most recent call last):
          File "/.../python3.5/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
            result = g.send(result)
          File "/.../python3.5/site-packages/buildbot/www/auth.py", line 131, in maybeAutoLogin
            yield self.updateUserInfo(request)
          File "/.../python3.5/site-packages/twisted/internet/defer.py", line 1613, in unwindGenerator 
            return _cancellableInlineCallbacks(gen)
          File "/.../python3.5/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
            _inlineCallbacks(None, g, status)
        --- <exception caught here> ---
          File "/.../python3.5/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/.../python3.5/site-packages/buildbot/www/resource.py", line 92, in failHttpError
            f.trap(Error)
          File "/.../python3.5/site-packages/twisted/python/failure.py", line 439, in trap
            self.raiseException()
          File "/.../python3.5/site-packages/twisted/python/failure.py", line 467, in raiseException
            raise self.value.with_traceback(self.tb)
          File "/.../python3.5/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/.../python3.5/site-packages/buildbot/www/resource.py", line 85, in failHttpRedirect
            f.trap(Redirect)
          File "/.../python3.5/site-packages/twisted/python/failure.py", line 439, in trap
            self.raiseException()
          File "/.../python3.5/site-packages/twisted/python/failure.py", line 467, in raiseException
            raise self.value.with_traceback(self.tb)
          File "/.../python3.5/site-packages/buildbot/www/config.py", line 126, in renderIndex
            yield self.config['auth'].maybeAutoLogin(request)
          File "/.../python3.5/site-packages/buildbot/www/auth.py", line 131, in maybeAutoLogin
            yield self.updateUserInfo(request)
          File "/.../python3.5/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
            result = g.send(result)
          File "/.../python3.5/site-packages/buildbot/www/auth.py", line 78, in updateUserInfo
            session.updateSession(request)
          File "/.../python3.5/site-packages/buildbot/www/service.py", line 110, in updateSession
            request.addCookie(cookiename, self.uid, path=b"/",
          File "/.../python3.5/site-packages/buildbot/www/service.py", line 137, in uid
            return jwt.encode(claims, self.site.session_secret, algorithm=SESSION_SECRET_ALGORITHM)
          File "/.../python3.5/site-packages/jwt/api_jwt.py", line 62, in encode
            cls=json_encoder
          File "/usr/lib/python3.5/json/__init__.py", line 237, in dumps
            **kw).encode(obj)
          File "/usr/lib/python3.5/json/encoder.py", line 198, in encode
            chunks = self.iterencode(o, _one_shot=True)
          File "/usr/lib/python3.5/json/encoder.py", line 256, in iterencode
            return _iterencode(o, 0)
          File "/usr/lib/python3.5/json/encoder.py", line 179, in default
            raise TypeError(repr(o) + " is not JSON serializable")
        builtins.TypeError: b'foobar' is not JSON serializable
```
following line [here](https://github.com/buildbot/buildbot/blob/master/master/buildbot/www/auth.py#L127) resolves the problem:
```python
user_info = {k: buildbot.util.bytes2unicode(v) for k, v in res.groupdict().items()}
```
Sorry if this is a duplicate, but I failed to find a ticket for this issue. 
I will create a PR as soon as I have spare time.


Discussion:
I have the same issue. However, for me the line above does not fix the problem.
In my case, inserting the following line after [this one](https://github.com/buildbot/buildbot/blob/master/master/buildbot/www/auth.py#L129) fixes it:

```python
session.user_info = {k: bytes2unicode(v) for k, v in session.user_info.items()}
```
I extracted line out of my method and didn't pay attention that I slightly refactored the method. Sorry for that. Here is my code:
```python
    def maybeAutoLogin(self, request):
        header = request.getHeader(self.header)
        if header is None:
            raise error.Error(
                403,
                b"missing http header " + self.header + b". Check your reverse proxy config!"
            )
        res = self.headerRegex.match(header)
        if res is None:
            raise error.Error(
                403,
                b'http header does not match regex! "' + header + b'" not matching ' + self.headerRegex.pattern
            )
        session = request.getSession()
        user_info = {k: buildbot.util.bytes2unicode(v) for k, v in res.groupdict().items()}
        if session.user_info != user_info:
            session.user_info = user_info
            yield self.updateUserInfo(request)
```

@atomikin Can you submit your change as a pull request to buildbot?  I think your approach is correct.
These bytes/unicode issues on Python 3 are a headache to track down, so I appreciate your efforts on this.

Resolving patch:
diff --git a/master/buildbot/www/auth.py b/master/buildbot/www/auth.py
--- a/master/buildbot/www/auth.py
+++ b/master/buildbot/www/auth.py
@@ -123,8 +123,9 @@ def maybeAutoLogin(self, request):
             raise Error(
                 403, b'http header does not match regex! "' + header + b'" not matching ' + self.headerRegex.pattern)
         session = request.getSession()
-        if session.user_info != dict(res.groupdict()):
-            session.user_info = dict(res.groupdict())
+        user_info = {k: bytes2unicode(v) for k, v in res.groupdict().items()}
+        if session.user_info != user_info:
+            session.user_info = user_info
             yield self.updateUserInfo(request)
 
 


Test patch:
diff --git a/master/buildbot/test/unit/test_www_auth.py b/master/buildbot/test/unit/test_www_auth.py
--- a/master/buildbot/test/unit/test_www_auth.py
+++ b/master/buildbot/test/unit/test_www_auth.py
@@ -114,9 +114,9 @@ def test_maybeAutoLogin(self):
         self.request.input_headers[b'HDR'] = b'rachel@foo.com'
         yield self.auth.maybeAutoLogin(self.request)
         self.assertEqual(self.request.session.user_info, {
-                         'username': b'rachel',
-                         'realm': b'foo.com',
-                         'email': b'rachel'})
+                         'username': 'rachel',
+                         'realm': 'foo.com',
+                         'email': 'rachel'})
 
     @defer.inlineCallbacks
     def test_maybeAutoLogin_no_header(self):