Repository: Mailu/Mailu

Issue:
Attempting to add an emoji URL in the admin interface results in a 500

## Before you open your issue
- [x] Check if no issue or pull-request for this already exists.
- [x] Check [documentation](https://mailu.io/master/) and [FAQ](https://mailu.io/master/faq.html). (Tip, use the search function on the documentation page)
- [x] You understand `Mailu` is made by volunteers in their **free time** — be conscise, civil and accept that delays can occur.
- [x] The title of the issue should be short and simple. It should contain specific terms related to the actual issue. Be specific while writing the title.

## Environment & Versions
### Environment
 - [x] docker-compose

### Versions
1.7

## Description
Attempting to add the domain 💜🏳️‍⚧️.ws (or in punycode xn--c8h5289nqoa.ws) results in the web interface returning a 500 status code with the domain not being added.

## Replication Steps
Try adding the domain  ```xn--c8h5289nqoa.ws``` in the admin interface

## Expected behaviour
A 200 status code is returned and the domain is added

## Logs
````
idna.core.InvalidCodepoint: Codepoint U+1F49C at position 1 of '💜🏳⚧' not allowed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python3.7/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/usr/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib/python3.7/site-packages/flask_login/utils.py", line 261, in decorated_view
    return func(*args, **kwargs)
  File "/app/mailu/ui/access.py", line 36, in wrapper
    return callback(function, args, kwargs, (), {})
  File "/app/mailu/ui/access.py", line 19, in callback
    return function(*args, **kwargs)
  File "/app/mailu/ui/views/domains.py", line 22, in domain_create
    conflicting_domain = models.Domain.query.get(form.name.data)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 959, in get
    return self._get_impl(ident, loading.load_on_pk_identity)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 1069, in _get_impl
    return db_load_fn(self, primary_key_identity)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 282, in load_on_pk_identity
    return q.one()
  File "/usr/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3275, in one
    ret = self.one_or_none()
  File "/usr/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3244, in one_or_none
    ret = list(self)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3317, in __iter__
    return self._execute_and_instances(context)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3342, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
    distilled_params,
  File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    e, util.text_type(statement), parameters, None, None
  File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1179, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 719, in _init_compiled
    param.append(processors[key](compiled_params[key]))
  File "/usr/lib/python3.7/site-packages/sqlalchemy/sql/type_api.py", line 1201, in process
    return process_param(value, dialect)
  File "/app/mailu/models.py", line 30, in process_bind_param
    return idna.encode(value).decode("ascii").lower()
  File "/usr/lib/python3.7/site-packages/idna/core.py", line 358, in encode
    s = alabel(label)
  File "/usr/lib/python3.7/site-packages/idna/core.py", line 270, in alabel
    ulabel(label)
  File "/usr/lib/python3.7/site-packages/idna/core.py", line 308, in ulabel
    check_label(label)
  File "/usr/lib/python3.7/site-packages/idna/core.py", line 261, in check_label
    raise InvalidCodepoint('Codepoint {0} at position {1} of {2} not allowed'.format(_unot(cp_value), pos+1, repr(label)))
sqlalchemy.exc.StatementError: (idna.core.InvalidCodepoint) Codepoint U+1F49C at position 1 of '💜🏳⚧' not allowed
[SQL: SELECT domain.created_at AS domain_created_at, domain.updated_at AS domain_updated_at, domain.comment AS domain_comment, domain.name AS domain_name, domain.max_users AS domain_max_users, domain.max_aliases AS domain_max_aliases, domain.max_quota_bytes AS domain_max_quota_bytes, domain.signup_enabled AS domain_signup_enabled 
FROM domain 
WHERE domain.name = ?]
[parameters: [{'%(140210145585744 param)s': 'xn--c8h5289nqoa.ws'}]]
$myIP - - [17/Jan/2021:21:16:17 +0000] "POST /ui/domain/create HTTP/1.1" 500 290 "https://$myDomain/admin/ui/domain/create" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:81.0) Gecko/20100101 Firefox/81.0"
````



Discussion:
Hi @DysphoricUnicorn. I've already fixed this in my unfinished fork/PR for the import/export function.
`.lower()` needs to be called _before_ punycode encoding:
`return idna.encode(value).decode("ascii").lower()` needs to be replaced with `return idna.encode(value.lower()).decode("ascii")` in models.py line 30.
Alex


Resolving patch:
diff --git a/core/admin/mailu/manage.py b/core/admin/mailu/manage.py
--- a/core/admin/mailu/manage.py
+++ b/core/admin/mailu/manage.py
@@ -386,7 +386,3 @@ def setmanager(domain_name, user_name='manager'):
     domain.managers.append(manageruser)
     db.session.add(domain)
     db.session.commit()
-
-
-if __name__ == '__main__':
-    cli()
diff --git a/core/admin/mailu/models.py b/core/admin/mailu/models.py
--- a/core/admin/mailu/models.py
+++ b/core/admin/mailu/models.py
@@ -15,6 +15,7 @@
 import smtplib
 import idna
 import dns
+import json
 
 
 db = flask_sqlalchemy.SQLAlchemy()
@@ -27,7 +28,7 @@ class IdnaDomain(db.TypeDecorator):
     impl = db.String(80)
 
     def process_bind_param(self, value, dialect):
-        return idna.encode(value).decode("ascii").lower()
+        return idna.encode(value.lower()).decode('ascii')
 
     def process_result_value(self, value, dialect):
         return idna.decode(value)


Test patch: