Repository: Mailu/Mailu

Issue:
No special characters in passwords allowed
<!--

Thank you for opening an issue with Mailu. Please understand that issues are meant for bugs and enhancement-requests.
For **user-support questions**, reach out to us  on [matrix](https://matrix.to/#/#mailu:tedomum.net).

To be able to help you best, we need some more information.

Before you open your issue
- Check if no issue or pull-request for this already exists.
- Check [documentation](https://mailu.io/master/) and [FAQ](https://mailu.io/master/faq.html). (Tip, use the search function on the documentation page)
- You understand `Mailu` is made by volunteers in their **free time** — be concise, civil and accept that delays can occur.
- The title of the issue should be short and simple. It should contain specific terms related to the actual issue. Be specific while writing the title.

Please put your text outside of the comment blocks to be visible. You can use the button "Preview" above to check.

-->

## Environment & Version

### Environment

- [x] docker compose
- [ ] kubernetes
- [ ] docker swarm

### Version

- Version: `2.0`

## Description

Authentication fails when using special characters in a user's password.

I cannot use special characters in a password, e.g. '€'. It worked in version 1.9

## Replication Steps

Use for example '€' in a user's password.

## Observed behaviour

Clients are not able to login to IMAP server.

## Expected behaviour

Clients should be able to login to IMAP server.

## Logs

Logs from `admin` container:

```
192.168.203.1 - - [30/May/2023:08:39:07 +0000] "GET /internal/auth/email HTTP/1.0" 500 0 "-" "-"
[2023-05-30 08:39:07 +0000] [18] [ERROR] Error handling request /internal/auth/email
Traceback (most recent call last):
  File "/app/venv/lib/python3.10/site-packages/gunicorn/workers/gthread.py", line 271, in handle
    keepalive = self.handle_request(req, conn)
  File "/app/venv/lib/python3.10/site-packages/gunicorn/workers/gthread.py", line 331, in handle_request
    resp.close()
  File "/app/venv/lib/python3.10/site-packages/gunicorn/http/wsgi.py", line 391, in close
    self.send_headers()
  File "/app/venv/lib/python3.10/site-packages/gunicorn/http/wsgi.py", line 322, in send_headers
    util.write(self.sock, util.to_bytestring(header_str, "latin-1"))
  File "/app/venv/lib/python3.10/site-packages/gunicorn/util.py", line 565, in to_bytestring
    return value.encode(encoding)
UnicodeEncodeError: 'latin-1' codec can't encode character '\u20ac' in position 267: ordinal not in range(256)
```


Discussion:
I can only replicate this for imap/smtp login (so login for email clients).
Login via the SSO form (Web) works fine.
I found the problematic part:
https://github.com/Mailu/Mailu/blob/097bad138f35c92823b65488066abc5b97e4a125/core/admin/mailu/internal/nginx.py#L114

If i comment this line, then the error does not occur. I guess this must be encoded?

Resolving patch:
diff --git a/core/admin/mailu/internal/nginx.py b/core/admin/mailu/internal/nginx.py
--- a/core/admin/mailu/internal/nginx.py
+++ b/core/admin/mailu/internal/nginx.py
@@ -111,7 +111,6 @@ def handle_authentication(headers):
                         "Auth-Server": server,
                         "Auth-User": user_email,
                         "Auth-User-Exists": is_valid_user,
-                        "Auth-Password": password,
                         "Auth-Port": port
                     }
         status, code = get_status(protocol, "authentication")
@@ -120,7 +119,6 @@ def handle_authentication(headers):
             "Auth-Error-Code": code,
             "Auth-User": user_email,
             "Auth-User-Exists": is_valid_user,
-            "Auth-Password": password,
             "Auth-Wait": 0
         }
     # Unexpected
diff --git a/core/admin/mailu/internal/views/auth.py b/core/admin/mailu/internal/views/auth.py
--- a/core/admin/mailu/internal/views/auth.py
+++ b/core/admin/mailu/internal/views/auth.py
@@ -6,6 +6,7 @@
 import flask_login
 import base64
 import sqlalchemy.exc
+import urllib
 
 @internal.route("/auth/email")
 def nginx_authentication():
@@ -30,6 +31,7 @@ def nginx_authentication():
         if int(flask.request.headers['Auth-Login-Attempt']) < 10:
             response.headers['Auth-Wait'] = '3'
         return response
+    raw_password = urllib.parse.unquote(headers["Auth-Pass"])
     headers = nginx.handle_authentication(flask.request.headers)
     response = flask.Response()
     for key, value in headers.items():
@@ -52,7 +54,14 @@ def nginx_authentication():
         if not is_port_25:
             utils.limiter.exempt_ip_from_ratelimits(client_ip)
     elif is_valid_user:
-        utils.limiter.rate_limit_user(username, client_ip, password=response.headers.get('Auth-Password', None))
+        password = None
+        try:
+            password = raw_password.encode("iso8859-1").decode("utf8")
+        except:
+            app.logger.warn(f'Received undecodable password for {username} from nginx: {raw_password!r}')
+            utils.limiter.rate_limit_user(username, client_ip, password=None)
+        else:
+            utils.limiter.rate_limit_user(username, client_ip, password=password)
     elif not is_from_webmail:
         utils.limiter.rate_limit_ip(client_ip, username)
     return response


Test patch:
diff --git a/tests/compose/core/00_create_users.sh b/tests/compose/core/00_create_users.sh
--- a/tests/compose/core/00_create_users.sh
+++ b/tests/compose/core/00_create_users.sh
@@ -8,4 +8,5 @@ docker compose -f tests/compose/core/docker-compose.yml exec -T admin flask mail
 docker compose -f tests/compose/core/docker-compose.yml exec -T admin flask mailu admin admin mailu.io 'password' --mode=update || exit 1
 docker compose -f tests/compose/core/docker-compose.yml exec -T admin flask mailu user user mailu.io 'password' || exit 1
 docker compose -f tests/compose/core/docker-compose.yml exec -T admin flask mailu user 'user/with/slash' mailu.io 'password' || exit 1
+docker compose -f tests/compose/core/docker-compose.yml exec -T admin flask mailu user 'user_UTF8' mailu.io 'password€' || exit 1
 echo "User testing successful!"
diff --git a/tests/compose/core/05_connectivity.py b/tests/compose/core/05_connectivity.py
--- a/tests/compose/core/05_connectivity.py
+++ b/tests/compose/core/05_connectivity.py
@@ -7,25 +7,31 @@
 import managesieve
 
 SERVER='localhost'
-USERNAME='user@mailu.io'
-PASSWORD='password'
+USERNAME='user_UTF8@mailu.io'
+PASSWORD='password€'
+#https://github.com/python/cpython/issues/73936
+#SMTPlib does not support UTF8 passwords.
+USERNAME_ASCII='user@mailu.io'
+PASSWORD_ASCII='password'
+
 
 def test_imap(server, username, password):
+    auth = lambda data : f'\x00{username}\x00{password}'
     print(f'Authenticating to imaps://{username}:{password}@{server}:993/')
     with imaplib.IMAP4_SSL(server) as conn:
-        conn.login(username, password)
+        conn.authenticate('PLAIN', auth)
         conn.noop()
     print('OK')
     print(f'Authenticating to imaps://{username}:{password}@{server}:143/')
     with imaplib.IMAP4(server) as conn:
         conn.starttls()
-        conn.login(username, password)
+        conn.authenticate('PLAIN', auth)
         conn.noop()
     print('OK')
     print(f'Authenticating to imap://{username}:{password}@{server}:143/')
     try:
         with imaplib.IMAP4(server) as conn:
-            conn.login(username, password)
+            conn.authenticate('PLAIN', auth)
         print(f'Authenticating to imap://{username}:{password}@{server}:143/ worked without STARTTLS!')
         sys.exit(102)
     except imaplib.IMAP4.error:
@@ -121,7 +127,7 @@ def test_managesieve(server, username, password):
     if m.login('', username, password) != 'OK':
         print(f'Authenticating to sieve://{username}:{password}@{server}:4190/ has failed!')
         sys.exit(109)
-    
+
     if m.listscripts()[0] != 'OK':
         print(f'Listing scripts failed!')
         sys.exit(110)
@@ -130,5 +136,7 @@ def test_managesieve(server, username, password):
 if __name__ == '__main__':
     test_imap(SERVER, USERNAME, PASSWORD)
     test_pop3(SERVER, USERNAME, PASSWORD)
-    test_SMTP(SERVER, USERNAME, PASSWORD)
+    test_SMTP(SERVER, USERNAME_ASCII, PASSWORD_ASCII)
     test_managesieve(SERVER, USERNAME, PASSWORD)
+#https://github.com/python/cpython/issues/73936
+#SMTPlib does not support UTF8 passwords.
\ No newline at end of file