internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: Mailu/Mailu

Issue:
Roundcube managesieve fails on IPv6
I have assigned a globally-routable IPv6 network to the docker-compose stack. Rainloop sieve management works correctly but Roundcube fails. Looking at the logs, Roundcube attempts to connect to sieve via IPv6. Hard-coding $imap to the IPv4 address of the imap docker container allows Roundcube sieve to function properly. I believe it is related to this in the debug log of imap:

`DEBUG:root:Table get passdb/[username redated] is {'allow_nets': '192.168.203.0/24', 'nopassword': 'Y', 'password': None}`

Using version 1.7.



Discussion:
My current workaround is setting the POD_ADDRESS_RANGE to the same value as SUBNET6.
Ignore that. That breaks SMTP.

Resolving patch:
diff --git a/core/admin/mailu/configuration.py b/core/admin/mailu/configuration.py
--- a/core/admin/mailu/configuration.py
+++ b/core/admin/mailu/configuration.py
@@ -67,6 +67,7 @@
     'HOST_REDIS': 'redis',
     'HOST_FRONT': 'front',
     'SUBNET': '192.168.203.0/24',
+    'SUBNET6': None,
     'POD_ADDRESS_RANGE': None
 }
 
diff --git a/core/admin/mailu/internal/views/dovecot.py b/core/admin/mailu/internal/views/dovecot.py
--- a/core/admin/mailu/internal/views/dovecot.py
+++ b/core/admin/mailu/internal/views/dovecot.py
@@ -11,6 +11,8 @@ def dovecot_passdb_dict(user_email):
     user = models.User.query.get(user_email) or flask.abort(404)
     allow_nets = []
     allow_nets.append(app.config["SUBNET"])
+    if app.config["SUBNET6"]:
+        allow_nets.append(app.config["SUBNET6"])
     if app.config["POD_ADDRESS_RANGE"]:
         allow_nets.append(app.config["POD_ADDRESS_RANGE"])
     return flask.jsonify({


Test patch:
Repository: Mailu/Mailu

Issue:
Add ability to specify SMTP port
Some relayed domains may live on machines that accept SMTP on a nonstandard port, e.g., port 2525. This would allow users to relay for domains by appending `:port` to the hostname or IP address.
Add ability to specify SMTP port
Some relayed domains may live on machines that accept SMTP on a nonstandard port, e.g., port 2525. This would allow users to relay for domains by appending `:port` to the hostname or IP address.


Discussion:

Resolving patch:
diff --git a/core/admin/mailu/internal/views/postfix.py b/core/admin/mailu/internal/views/postfix.py
--- a/core/admin/mailu/internal/views/postfix.py
+++ b/core/admin/mailu/internal/views/postfix.py
@@ -36,7 +36,11 @@ def postfix_transport(email):
         return flask.abort(404)
     localpart, domain_name = models.Email.resolve_domain(email)
     relay = models.Relay.query.get(domain_name) or flask.abort(404)
-    return flask.jsonify("smtp:[{}]".format(relay.smtp))
+    ret = "smtp:[{0}]".format(relay.smtp)
+    if ":" in relay.smtp:
+        split = relay.smtp.split(':')
+        ret = "smtp:[{0}]:{1}".format(split[0], split[1])
+    return flask.jsonify(ret)
 
 
 @internal.route("/postfix/sender/login/<path:sender>")


Test patch:
Repository: Mailu/Mailu

Issue:
Case insensitiveness broken for aliases
If there’s a catchall (`foo%`) as well as a specific alias (`foobar`) on a domain, and a mail arrives for that alias, but in a different case (`FOOBAR`), it is delivered to the catchall instead of the specific alias. If the mail arrives in the correct case, the specific alias is chosen. This is inconsistent behavior (… and endlessly unnerving with airlines and the likes), so brace for a hopefully-soon-to-come PR ^_^.


Discussion:

Resolving patch:
diff --git a/core/admin/mailu/models.py b/core/admin/mailu/models.py
--- a/core/admin/mailu/models.py
+++ b/core/admin/mailu/models.py
@@ -451,25 +451,34 @@ def resolve(cls, localpart, domain_name):
                     )
                 )
             ).order_by(cls.wildcard, sqlalchemy.func.char_length(cls.localpart).desc()).first()
-        if alias_preserve_case:
-            return alias_preserve_case
 
-        if localpart:
-            localpart = localpart.lower()
-        return cls.query.filter(
+        localpart_lower = localpart.lower() if localpart else None
+        alias_lower_case = cls.query.filter(
                 sqlalchemy.and_(cls.domain_name == domain_name,
                     sqlalchemy.or_(
                         sqlalchemy.and_(
                             cls.wildcard == False,
-                            sqlalchemy.func.lower(cls.localpart) == localpart
+                            sqlalchemy.func.lower(cls.localpart) == localpart_lower
                         ), sqlalchemy.and_(
                             cls.wildcard == True,
-                            sqlalchemy.bindparam("l", localpart).like(sqlalchemy.func.lower(cls.localpart))
+                            sqlalchemy.bindparam("l", localpart_lower).like(sqlalchemy.func.lower(cls.localpart))
                         )
                     )
                 )
             ).order_by(cls.wildcard, sqlalchemy.func.char_length(sqlalchemy.func.lower(cls.localpart)).desc()).first()
 
+        if alias_preserve_case and alias_lower_case:
+            if alias_preserve_case.wildcard:
+                return alias_lower_case
+            else:
+                return alias_preserve_case
+        elif alias_preserve_case and not alias_lower_case:
+            return alias_preserve_case
+        elif alias_lower_case and not alias_preserve_case:
+            return alias_lower_case
+        else:
+            return None
+
 class Token(Base):
     """ A token is an application password for a given user.
     """


Test patch:
Repository: Mailu/Mailu

Issue:
Postconf overrides should ignore comments
Comment lines in overrides/postfix.cf are sent to postconf in core/postfix/start.py:37. This results in error messages.

Comment lines should be ignored by core/postfix/start.py.


Discussion:
Also empty newlines as well.

Resolving patch:
diff --git a/core/postfix/start.py b/core/postfix/start.py
--- a/core/postfix/start.py
+++ b/core/postfix/start.py
@@ -27,6 +27,10 @@ def start_podop():
         ("senderlogin", "url", url + "sender/login/§")
     ])
 
+def is_valid_postconf_line(line):
+    return not line.startswith("#") \
+            and not line == ''
+
 # Actual startup script
 os.environ["FRONT_ADDRESS"] = system.get_host_address_from_environment("FRONT", "front")
 os.environ["ADMIN_ADDRESS"] = system.get_host_address_from_environment("ADMIN", "admin")
@@ -38,11 +42,13 @@ def start_podop():
 
 if os.path.exists("/overrides/postfix.cf"):
     for line in open("/overrides/postfix.cf").read().strip().split("\n"):
-        os.system('postconf -e "{}"'.format(line))
+        if is_valid_postconf_line(line):
+            os.system('postconf -e "{}"'.format(line))
 
 if os.path.exists("/overrides/postfix.master"):
     for line in open("/overrides/postfix.master").read().strip().split("\n"):
-        os.system('postconf -Me "{}"'.format(line))
+        if is_valid_postconf_line(line):
+            os.system('postconf -Me "{}"'.format(line))
 
 for map_file in glob.glob("/overrides/*.map"):
     destination = os.path.join("/etc/postfix", os.path.basename(map_file))


Test patch:
Repository: Mailu/Mailu

Issue:
Default yaml.load deprecated and unsafe
I've come across the following warnings in the [travis logs](https://travis-ci.org/Mailu/Mailu/jobs/563724185#L3319):

```
/app/mailu/manage.py:158: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
```
This refers to the following snippet:

https://github.com/Mailu/Mailu/blob/2788909a134217f2e5516fff938df55b644c258c/core/admin/mailu/manage.py#L158

Upstream notice:
https://msg.pyyaml.org/load

And a related CVE:
https://nvd.nist.gov/vuln/detail/CVE-2017-18342



Discussion:

Resolving patch:
diff --git a/core/admin/mailu/manage.py b/core/admin/mailu/manage.py
--- a/core/admin/mailu/manage.py
+++ b/core/admin/mailu/manage.py
@@ -177,7 +177,7 @@ def config_update(verbose=False, delete_objects=False):
     """sync configuration with data from YAML-formatted stdin"""
     import yaml
     import sys
-    new_config = yaml.load(sys.stdin)
+    new_config = yaml.safe_load(sys.stdin)
     # print new_config
     domains = new_config.get('domains', [])
     tracked_domains = set()


Test patch:
Repository: Mailu/Mailu

Issue:
We need to check the (persistent) postfix mailqueue permissions
As the postfix and postdrop uid/gid might change (especially with a change of base image), it is needed to check the directory permissions on /queue before starting postfix


Discussion:

Resolving patch:
diff --git a/core/postfix/start.py b/core/postfix/start.py
--- a/core/postfix/start.py
+++ b/core/postfix/start.py
@@ -64,4 +64,7 @@ def is_valid_postconf_line(line):
 # Run Podop and Postfix
 multiprocessing.Process(target=start_podop).start()
 os.system("/usr/libexec/postfix/post-install meta_directory=/etc/postfix create-missing")
+# Before starting postfix, we need to check permissions on /queue
+# in the event that postfix,postdrop id have changed
+os.system("postfix set-permissions")
 os.system("postfix start-fg")


Test patch:
Repository: Mailu/Mailu

Issue:
Dovecot does not use redis, so it should be removed from start script 
In core/dovecot/start.py REDIS_ADDRESS is resolved but redis is not used on dovecot. It should be removed from the script.


Discussion:
Same for WEBMAIL and WEBMAIL_ADDRESS
~Same for ADMIN_ADDRESS~ no, this is required ;)

Resolving patch:
diff --git a/core/dovecot/start.py b/core/dovecot/start.py
--- a/core/dovecot/start.py
+++ b/core/dovecot/start.py
@@ -21,13 +21,9 @@ def start_podop():
     ])
 
 # Actual startup script
-
 os.environ["FRONT_ADDRESS"] = system.get_host_address_from_environment("FRONT", "front")
-os.environ["REDIS_ADDRESS"] = system.get_host_address_from_environment("REDIS", "redis")
 os.environ["ADMIN_ADDRESS"] = system.get_host_address_from_environment("ADMIN", "admin")
 os.environ["ANTISPAM_WEBUI_ADDRESS"] = system.get_host_address_from_environment("ANTISPAM_WEBUI", "antispam:11334")
-if os.environ["WEBMAIL"] != "none":
-    os.environ["WEBMAIL_ADDRESS"] = system.get_host_address_from_environment("WEBMAIL", "webmail")
 
 for dovecot_file in glob.glob("/conf/*.conf"):
     conf.jinja(dovecot_file, os.environ, os.path.join("/etc/dovecot", os.path.basename(dovecot_file)))


Test patch:
Repository: Mailu/Mailu

Issue:
Hardcoded http://admin/ in fetchmail.py
I've tweaked ``docker-compose.yml`` so that all my containers related to ``mailu`` are prefixed by ``mailu-``, in order to pro-actively avoid conflict with any other containers I may eventually define in future.

However, the hardcode ``http://admin/...`` below causes failure in ``fetchmail``, since my container is now named ``mailu-admin`` in my ``docker-compose.yml``, not ``admin`` as the code supposes it should be.

```
./services/fetchmail/fetchmail.py:47:    fetches = requests.get("http://admin/internal/fetch").json()
./services/fetchmail/fetchmail.py:85:            requests.post("http://admin/internal/fetch/{}".format(fetch["id"]),
```


Discussion:

Resolving patch:
diff --git a/optional/fetchmail/fetchmail.py b/optional/fetchmail/fetchmail.py
--- a/optional/fetchmail/fetchmail.py
+++ b/optional/fetchmail/fetchmail.py
@@ -47,7 +47,7 @@ def fetchmail(fetchmailrc):
 
 def run(debug):
     try:
-        fetches = requests.get("http://admin/internal/fetch").json()
+        fetches = requests.get("http://" + os.environ.get("HOST_ADMIN", "admin") + "/internal/fetch").json()
         smtphost, smtpport = extract_host_port(os.environ.get("HOST_SMTP", "smtp"), None)
         if smtpport is None:
             smtphostport = smtphost
@@ -85,7 +85,7 @@ def run(debug):
                         user_info in error_message):
                     print(error_message)
             finally:
-                requests.post("http://admin/internal/fetch/{}".format(fetch["id"]),
+                requests.post("http://" + os.environ.get("HOST_ADMIN", "admin") + "/internal/fetch/{}".format(fetch["id"]),
                     json=error_message.split("\n")[0]
                 )
     except Exception:


Test patch:
Repository: Mailu/Mailu

Issue:
Remove setup stack "replicas" option
In preparation for 1.8-rc release, and according to #1582 , remove the "replicas" option from setup step 5 in the stack deployment option. Also add an additional notice that users should review their configuration by hand, as well as a big warning about "replicas prone to data-corruption" to the "experimental" box at the top of setup.
(note that replicas is just *currently* not supported.)

Note that if you know what you’re doing, replicas can be easily configured by hand.

Relates: #1222 #1224 #1637


Discussion:

Resolving patch:
diff --git a/core/admin/mailu/configuration.py b/core/admin/mailu/configuration.py
--- a/core/admin/mailu/configuration.py
+++ b/core/admin/mailu/configuration.py
@@ -24,7 +24,7 @@
     'SQLALCHEMY_TRACK_MODIFICATIONS': False,
     # Statistics management
     'INSTANCE_ID_PATH': '/data/instance',
-    'STATS_ENDPOINT': '0.{}.stats.mailu.io',
+    'STATS_ENDPOINT': '18.{}.stats.mailu.io',
     # Common configuration variables
     'SECRET_KEY': 'changeMe',
     'DOMAIN': 'mailu.io',
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -41,6 +41,7 @@
         ('1.5', '/1.5/'),
         ('1.6', '/1.6/'),
         ('1.7', '/1.7/'),
+        ('1.8', '/1.8/'),
         ('master', '/master/')
     ],
     'conf_py_path': '/docs/'
diff --git a/setup/server.py b/setup/server.py
--- a/setup/server.py
+++ b/setup/server.py
@@ -53,7 +53,10 @@ def build_app(path):
 
     @app.context_processor
     def app_context():
-        return dict(versions=os.getenv("VERSIONS","master").split(','))
+        return dict(
+            versions=os.getenv("VERSIONS","master").split(','), 
+            stable_version = os.getenv("stable_version", "master")
+        )
 
     prefix_bp = flask.Blueprint(version, __name__)
     prefix_bp.jinja_loader = jinja2.ChoiceLoader([


Test patch:
Repository: Mailu/Mailu

Issue:
Radicale password check fails (length limited?)
So far, the webdav radicale is protected with basic auth (apparently htaccess or similar).
If user password > 8 then it fails to connect. 
we should remove this limitation to let use stong passwords


Discussion:
As far as I know, it's handled in 
https://github.com/Mailu/Mailu/blob/66ea28b50adce1c589fd6b2dc1b34356d4c64930/core/admin/mailu/internal/views/auth.py#L60 and there is no length limit.

Which version have you tested with? What's in master? Is there anything logged in the admin container?

I have fixed a bug whereby tokens (that are longer) weren't considered... but that was not length related
I think we should update the basic_authentication method. The handling of the base64 encoded data IMHO is a little bit sloppy. 

@parisni  do you have a colon in the password? I think the split in line 66 will fail if there is any (", 1" missing).

The username must not contain a colon, so it should be .split(b':', 1)
@ghostwheel42 good catch! Yeah, I've checked the RFC and the username can't have a colon... maybe we should enforce that elsewhere too.
@nextgens 
> maybe we should enforce that elsewhere too.

There user name is a E-Mail address. The domain part is "checked" in the model: AFAIK idna encode does not encode a colon. 
I don't see a check for the local part, but this is part of my config-import/export/schema/api work in progress: I'll add validators to the model. 


Resolving patch:
diff --git a/core/admin/mailu/internal/views/auth.py b/core/admin/mailu/internal/views/auth.py
--- a/core/admin/mailu/internal/views/auth.py
+++ b/core/admin/mailu/internal/views/auth.py
@@ -51,7 +51,7 @@ def basic_authentication():
     authorization = flask.request.headers.get("Authorization")
     if authorization and authorization.startswith("Basic "):
         encoded = authorization.replace("Basic ", "")
-        user_email, password = base64.b64decode(encoded).split(b":")
+        user_email, password = base64.b64decode(encoded).split(b":", 1)
         user = models.User.query.get(user_email.decode("utf8"))
         if user and user.enabled and user.check_password(password.decode("utf8")):
             response = flask.Response()


Test patch:
Repository: Mailu/Mailu

Issue:
Override to use server hostname + IP in outbound headers
## What type of PR?
Enhancement

## What does this PR do?
Allows users to set the server IP and hostname in the outgoing "Received" header rather than "PUBLIC_HOSTNAME [PUBLIC_IP]".

### Related issue(s)
https://github.com/Mailu/Mailu/issues/191

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.



Discussion:
Thanks for submitting this pull request.
Bors-ng will now build test images. When it succeeds, we will continue to review and test your PR.

bors try

Note: if this build fails, [read this](http://mailu.io/master/contributors/environment.html#when-bors-try-fails).

## try

Build succeeded:
  * [continuous-integration/travis-ci/push](https://travis-ci.org/github/Mailu/Mailu/builds/674816408?utm_source=github_status&utm_medium=notification)
Does it make sense to generally use something like `os.environ["HOSTNAMES"].split(",")[0]` + 127.0.0.1/0.0.0.0/..?

For example Exchange Online also seems to just write the server handling the connection instead of the client into the header:

```
Received: from AM0PR05MB4452.eurprd05.prod.outlook.com
 ([fe80::51ec:24fe:245c:527e]) by AM0PR05MB4452.eurprd05.prod.outlook.com
 ([fe80::51ec:24fe:245c:527e%7]) with mapi id 15.20.3391.014; Wed, 16 Sep 2020
 08:08:02 +0000
```

The server information is public anyway and it's probably better than a generic placeholder.
I agree; I think it would make sense to avoid a new configuration flag... and to turn it on by default with a fallback to 127.0.0.1 if the address lookup fails.
@dmwilcox are you still okay to work on this?
Looking forward to see this feature ;)

Resolving patch:
diff --git a/core/postfix/start.py b/core/postfix/start.py
--- a/core/postfix/start.py
+++ b/core/postfix/start.py
@@ -36,6 +36,11 @@ def is_valid_postconf_line(line):
 os.environ["ADMIN_ADDRESS"] = system.get_host_address_from_environment("ADMIN", "admin")
 os.environ["ANTISPAM_MILTER_ADDRESS"] = system.get_host_address_from_environment("ANTISPAM_MILTER", "antispam:11332")
 os.environ["LMTP_ADDRESS"] = system.get_host_address_from_environment("LMTP", "imap:2525")
+os.environ["OUTCLEAN"] = os.environ["HOSTNAMES"].split(",")[0]
+try:
+    os.environ["OUTCLEAN_ADDRESS"] = system.resolve_hostname(os.environ["OUTCLEAN"])
+except:
+    os.environ["OUTCLEAN_ADDRESS"] = "10.10.10.10"
 
 for postfix_file in glob.glob("/conf/*.cf"):
     conf.jinja(postfix_file, os.environ, os.path.join("/etc/postfix", os.path.basename(postfix_file)))


Test patch:
Repository: Mailu/Mailu

Issue:
Weblate instance is down
I tried accessing the Weblate instance and potentially add another language but it looks down.


Discussion:

Resolving patch:
diff --git a/setup/server.py b/setup/server.py
--- a/setup/server.py
+++ b/setup/server.py
@@ -54,11 +54,11 @@ def build_app(path):
     @app.context_processor
     def app_context():
         return dict(
-            versions=os.getenv("VERSIONS","master").split(','), 
+            versions=os.getenv("VERSIONS","master").split(','),
             stable_version = os.getenv("stable_version", "master")
         )
 
-    prefix_bp = flask.Blueprint(version, __name__)
+    prefix_bp = flask.Blueprint(version.replace(".", "_"), __name__)
     prefix_bp.jinja_loader = jinja2.ChoiceLoader([
         jinja2.FileSystemLoader(os.path.join(path, "templates")),
         jinja2.FileSystemLoader(os.path.join(path, "flavors"))


Test patch:
Repository: Mailu/Mailu

Issue:
Postfix no longer starts correctly in kubernetes
## Environment & Versions
### Environment
 - [ ] docker-compose
 - [X] kubernetes
 - [ ] docker swarm

### Versions
1.8

## Description
After installing mailu 1.8 via helm on Kubernetes, the `mailu-postfix` container runs but never reaches a healthy state and smtp functionality is impaired.

After digging into it I believe the container is failing to become healthy because the following change to the postfix container's startup script (https://github.com/Mailu/Mailu/commit/1d65529c94f54de3cb49ed9584ed95f7860c26fa) and a known issue with the musl resolver the alpine base image uses (https://github.com/kubernetes/kubernetes/issues/64924).

Resolving the mailu installation hostname never succeeds because of the aforementioned bug, and `socrates.system.resolve_hostname` simply retries until the pod's failure threshold is exceeded and is restarted.

There's a couple different ways I believe this could be resolved:

1. Pass a FQDN to `system.resolve_hostname()`, which avoids the resolver bug with search lists, i.e. `domain.com.` with a trailing dot.

2. Update the deployment manifest in the mailu helm chart to use `dnsConfig.options` on the pod spec to set a more agreeable `ndots` value for `/etc/resolv.conf`

3. Use a different base image for mailu containers that is not affected by this issue.

I would be happy to investigate further and file a PR with the appropriate changes based on feedback. Thanks!


Discussion:

Resolving patch:
diff --git a/core/postfix/start.py b/core/postfix/start.py
--- a/core/postfix/start.py
+++ b/core/postfix/start.py
@@ -38,7 +38,11 @@ def is_valid_postconf_line(line):
 os.environ["LMTP_ADDRESS"] = system.get_host_address_from_environment("LMTP", "imap:2525")
 os.environ["OUTCLEAN"] = os.environ["HOSTNAMES"].split(",")[0]
 try:
-    os.environ["OUTCLEAN_ADDRESS"] = system.resolve_hostname(os.environ["OUTCLEAN"])
+    _to_lookup = os.environ["OUTCLEAN"]
+    # Ensure we lookup a FQDN: @see #1884
+    if not _to_lookup.endswith('.'):
+        _to_lookup += '.'
+    os.environ["OUTCLEAN_ADDRESS"] = system.resolve_hostname(_to_lookup)
 except:
     os.environ["OUTCLEAN_ADDRESS"] = "10.10.10.10"
 


Test patch: