internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: Mailu/Mailu

Issue:
DKIM keys aren't generated
Hello, 

I have set up mailu and it's generally working fine, I can add domains and users and receive mail, but I'm unable to proceed with DKIM keys generation.

My steps:
Click on "details" icon for the domain 
- I see a table with three lines: DNS MX entry, DNS SPF entries, DNS client auto-configuration (RFC6186) entries. NO DKIM

Click "Regenerate Keys"
- I see a confirmation prompt

Click "Confirm"
- A POST request is sent to https://mail.example.com/admin/ui/domain/genkeys/example.com and results in a 302 redirect back to the details page for the domain. But the details still show only the 3 above mentioned lines. DKIM keys do not appear. 

I don't know if it's relevant but the permissions to the /mailu/dkim directory on the host machine are

```
drwxr-xr-x  2 root   root     4096 Jul 24 17:51 dkim
```

I tried setting them temporarily to 777 but that didn't help so I restored them to 755. This directory remains empty before and after the request.

## Environment & Versions
### Environment
 - [x] docker-compose on Ubuntu 20.04

### Versions
Mailu version is 1.7 

## Logs
I've been watching the logs of the antispam and admin containers during the regeneration request. Neither container generates any new log lines at the moment of request. 

Please let me know how I can help debug the issue or if there's a command I can run in the container manually to generate keys for DKIM. Thanks!



Discussion:

Resolving patch:
diff --git a/core/admin/mailu/models.py b/core/admin/mailu/models.py
--- a/core/admin/mailu/models.py
+++ b/core/admin/mailu/models.py
@@ -272,7 +272,7 @@ def dkim_publickey(self):
             return dkim.strip_key(dkim_key).decode('utf8')
 
     def generate_dkim_key(self):
-        """ generate and activate new DKIM key """
+        """ generate new DKIM key """
         self.dkim_key = dkim.gen_key()
 
     def has_email(self, localpart):
diff --git a/core/admin/mailu/ui/views/domains.py b/core/admin/mailu/ui/views/domains.py
--- a/core/admin/mailu/ui/views/domains.py
+++ b/core/admin/mailu/ui/views/domains.py
@@ -74,6 +74,8 @@ def domain_details(domain_name):
 def domain_genkeys(domain_name):
     domain = models.Domain.query.get(domain_name) or flask.abort(404)
     domain.generate_dkim_key()
+    models.db.session.add(domain)
+    models.db.session.commit()
     return flask.redirect(
         flask.url_for(".domain_details", domain_name=domain_name))
 


Test patch:
Repository: Mailu/Mailu

Issue:
Rate limiting changes for 1.8
Copied from #1582. For 1.8 we will for now increase rate limiting value and disable rate limiting for the  subnet. 

- Rate limiting
  - Document rate limiting
  - Currently the subnet is included in the rate limiting. This means that a user who repeatly fails to login the webmail, blocks the webmail for ALL users.

- For 1.8 and master
  - in mailu.env set the rate limit to a high value.
  - in mailu.env disable the rate limiter for the subnet.
  - And document this of course and change this in the documentation
  - Set status blocked on lubs pull request and request to further discuss this for mailu 1.9.
  -  Make authentication fast #1745 is a draft pr from nextgens which contains a solution for this problem.
  -  We need a new issue and PR for making these changes.


Discussion:
Work to be done for the above:
- subnet rate limiting:
  - Change in docs/configuration.rst that AUTH_RATELIMIT_SUBNET is set to False by default.
  - Change in core/admin/mailu/configuration.py the default value of AUTH_RATELIMIT_SUBNET to False.
  - Triple check it is safe to backport this to 1.8. Otherwise this must be done in 2 separate PRs. One on branch 1.8 and one on branch master.

- higher default rate limit
  - Add to towncrier newsfragment that I must not forget to document this in the release notes (once 1.8 is released) that we increased the default value for rate limiting. 
  - Change in [config.html](https://github.com/Mailu/Mailu/blob/56a6821875a2742e18e087e681b97f6820edde9a/setup/templates/steps/config.html#L54) the default value of 10 to a high value.



I will change the default value to 10000/minute . I think this is high enough to prevent any issues.


Resolving patch:
diff --git a/core/admin/mailu/configuration.py b/core/admin/mailu/configuration.py
--- a/core/admin/mailu/configuration.py
+++ b/core/admin/mailu/configuration.py
@@ -34,8 +34,8 @@
     'POSTMASTER': 'postmaster',
     'TLS_FLAVOR': 'cert',
     'INBOUND_TLS_ENFORCE': False,
-    'AUTH_RATELIMIT': '10/minute;1000/hour',
-    'AUTH_RATELIMIT_SUBNET': True,
+    'AUTH_RATELIMIT': '1000/minute;10000/hour',
+    'AUTH_RATELIMIT_SUBNET': False,
     'DISABLE_STATISTICS': False,
     # Mail settings
     'DMARC_RUA': None,


Test patch:
Repository: Mailu/Mailu

Issue:
Optimize Rainloop image
## Before you open your issue
- [X] Check if no issue or pull-request for this already exists.
- [X] Check [documentation](https://mailu.io/master/) and [FAQ](https://mailu.io/master/faq.html). (Tip, use the search function on the documentation page)
- [X] You understand `Mailu` is made by volunteers in their **free time** — be conscise, civil and accept that delays can occur.
- [X] The title of the issue should be short and simple. It should contain specific terms related to the actual issue. Be specific while writing the title.

## Environment & Versions
### Environment
 - [X] docker-compose
 - [X] kubernetes
 - [X] docker swarm

### Versions
Master
```
$> docker images
mailu/rainloop               master            2ad8d1c29ff3   45 hours ago     607MB
```
Optimized version using NGINX:
```
user/rainloop                master            7de9dee9286d   2 seconds ago   116MB
```

## Description
In order to resolve issues #1830 and #1200, it is required to reduce build time, complexity and image size of images before multi-architecture builds can be supported. The current Rainloop image size is 607MB and can be optimized when changing from Apache to NGINX with officially supported Docker images. This can also increase overall web performance.


Discussion:

Resolving patch:
diff --git a/webmails/rainloop/start.py b/webmails/rainloop/start.py
--- a/webmails/rainloop/start.py
+++ b/webmails/rainloop/start.py
@@ -19,12 +19,11 @@
 os.makedirs(base + "domains", exist_ok=True)
 os.makedirs(base + "configs", exist_ok=True)
 
-conf.jinja("/default.ini", os.environ, "/data/_data_/_default_/domains/default.ini")
-conf.jinja("/application.ini", os.environ, "/data/_data_/_default_/configs/application.ini")
-conf.jinja("/php.ini", os.environ, "/usr/local/etc/php/conf.d/rainloop.ini")
+conf.jinja("/defaults/default.ini", os.environ, "/data/_data_/_default_/domains/default.ini")
+conf.jinja("/defaults/application.ini", os.environ, "/data/_data_/_default_/configs/application.ini")
+conf.jinja("/defaults/php.ini", os.environ, "/etc/php7/php.ini")
 
-os.system("chown -R www-data:www-data /data")
-os.system("chmod -R a+rX /var/www/html/")
-
-os.execv("/usr/local/bin/apache2-foreground", ["apache2-foreground"])
+os.system("chown -R nginx:nginx /data")
+os.system("chmod -R a+rX /var/www/rainloop/")
 
+os.execv("/usr/sbin/nginx", ["nginx", "-g", "daemon off;"])


Test patch:
Repository: Mailu/Mailu

Issue:
Reply to a letter from a subaddress
Now, if an email has been sent to a subaddress, for example, ```user+subadress@domain.tld``` and when you try to answer it, the address of the poisoner will be indicated ```user@domain.tld``` not ```user+subadress@domain.tld```. Is it possible to reply from the same address to which the letter was sent?


Discussion:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Hi. It is up to the user agent how to handle the subaddress part.
You should be able to configure multiple identities when using the optional webmail component of mailu. 
Alex
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

> Hi. It is up to the user agent how to handle the subaddress part. You should be able to configure multiple identities when using the optional webmail component of mailu. Alex

Can I please have a more detailed answer?
If something is required of me, I can also explain in more detail. Sorry, English is not my native language.
> Can I please have a more detailed answer?
> If something is required of me, I can also explain in more detail. Sorry, English is not my native language.

You just need to set "user+detail@domain.tld" as your sender address in your email-client.
But I've found a bug preventing you from sending emails this way. I'll fix it.

Resolving patch:
diff --git a/core/admin/mailu/configuration.py b/core/admin/mailu/configuration.py
--- a/core/admin/mailu/configuration.py
+++ b/core/admin/mailu/configuration.py
@@ -49,6 +49,7 @@
     'DKIM_PATH': '/dkim/{domain}.{selector}.key',
     'DEFAULT_QUOTA': 1000000000,
     'MESSAGE_RATELIMIT': '200/day',
+    'RECIPIENT_DELIMITER': '',
     # Web settings
     'SITENAME': 'Mailu',
     'WEBSITE': 'https://mailu.io',
diff --git a/core/admin/mailu/internal/views/postfix.py b/core/admin/mailu/internal/views/postfix.py
--- a/core/admin/mailu/internal/views/postfix.py
+++ b/core/admin/mailu/internal/views/postfix.py
@@ -140,6 +140,7 @@ def postfix_sender_login(sender):
     localpart, domain_name = models.Email.resolve_domain(sender)
     if localpart is None:
         return flask.jsonify(",".join(wildcard_senders)) if wildcard_senders else flask.abort(404)
+    localpart = localpart[:next((i for i, ch in enumerate(localpart) if ch in flask.current_app.config.get('RECIPIENT_DELIMITER')), None)]
     destination = models.Email.resolve_destination(localpart, domain_name, True)
     destination = [*destination, *wildcard_senders] if destination else [*wildcard_senders]
     return flask.jsonify(",".join(destination)) if destination else flask.abort(404)


Test patch:
Repository: Mailu/Mailu

Issue:
DKIM with multiple mail domains
I have a few small mail domains in mailu and I have setup DMARC records with rua and ruf mail addresses. The DMARC record vor domainA reads:

_dmarc.domainA.ch. 600 IN TXT "v=DMARC1; p=reject; rua=mailto:admin@smtp.masterdomain.ch; ruf=mailto:admin@smtp.masterdomain.ch; adkim=s; aspf=s"

However, this does not work unless there exists another DNS entry in the smtp.masterdomain.ch zone:

domainA.ch._report._dmarc.smtp.masterdomain.ch.    TXT    "v=DMARC1"
or
*._report._dmarc.smtp.masterdomain.ch.    TXT    "v=DMARC1"

When this entry is missing, some emails get rejected (sometimes), and flagged as 'dkim=fail'. I believe this is why I sometimes get mails rejected by hotmail.com.

Would it be possible to add these required DNS records to the Domain Details page /admin/ui/domain/details/domainA.ch ?

See also:
https://forum.directadmin.com/threads/how-to-setup-a-dns-record-to-recevie-report-for-dmarc-report-from-other-domains.56175/post-292355

Thanks,
Peter


Discussion:
Hi There,

The `Mailu`-Project is currently in a bit of a bind! We are short on man-power, and we need to judge if it is possible for us to put in some work on this issue.

To help with that, we are currently trying to find out which issues are actively keeping users from using `Mailu`, which issues have someone who want to work on them — and which issues may be less important. These a less important ones could be discarded for the time being, until the project is in a more stable and regular state once again.

In order for us to better assess this, it would be helpful if you could put a **reaction on this post** (use the :smiley: icon to the top-right).
- 👍️ if you **need this** to be able to use Mailu. Ideally, you’d also be able to test this on your installation, and provide feedback …
- 🎉 if you find it a **nice bonus**, but no deal-breaker
- 🚀 if you want to **work on it yourself**!
We want to keep this **voting open for 2 weeks** from now, so please help out!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

This should at least be added to the documentation. It would indeed be nice if this is displayed in the domain details page.

Resolving patch:
diff --git a/core/admin/mailu/models.py b/core/admin/mailu/models.py
--- a/core/admin/mailu/models.py
+++ b/core/admin/mailu/models.py
@@ -243,6 +243,13 @@ def dns_dmarc(self):
             ruf = f' ruf=mailto:{ruf}@{domain};' if ruf else ''
             return f'_dmarc.{self.name}. 600 IN TXT "v=DMARC1; p=reject;{rua}{ruf} adkim=s; aspf=s"'
 
+    @cached_property
+    def dns_dmarc_report(self):
+        """ return DMARC report record for mailu server """
+        if self.dkim_key:
+            domain = app.config['DOMAIN']
+            return f'{self.name}._report._dmarc.{domain}. 600 IN TXT "v=DMARC1"'
+
     @cached_property
     def dns_autoconfig(self):
         """ return list of auto configuration records (RFC6186) """


Test patch:
Repository: Mailu/Mailu

Issue:
Timezone in containers
Hello,

I have seen that timezones are not set for containers. This causes logs to have wrong timestamps and email "received" headers have wrong timezones.

It should be possible to set the timezone for all containers. The setting should be exposed and described in the .env file.


Discussion:
On my own server I passthrough the timezone to every container via volume:

```
    volumes:
      - "/etc/localtime:/etc/localtime:ro"
```

So I only have to take care of the timezone for host system. 
Hi There,

The `Mailu`-Project is currently in a bit of a bind! We are short on man-power, and we need to judge if it is possible for us to put in some work on this issue.

To help with that, we are currently trying to find out which issues are actively keeping users from using `Mailu`, which issues have someone who want to work on them — and which issues may be less important. These a less important ones could be discarded for the time being, until the project is in a more stable and regular state once again.

In order for us to better assess this, it would be helpful if you could put a **reaction on this post** (use the :smiley: icon to the top-right).
- 👍️ if you **need this** to be able to use Mailu. Ideally, you’d also be able to test this on your installation, and provide feedback …
- 🎉 if you find it a **nice bonus**, but no deal-breaker
- 🚀 if you want to **work on it yourself**!
We want to keep this **voting open for 2 weeks** from now, so please help out!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Thank you for your offer, @atl285 !
It looks like just mounting /etc/localtime does not do the job on recent alpine images. I also need to mount /usr/share/zoneinfo or alternatively install the tzdata package to make it work. Since that requires special knowledge, it's probably better to handle it within the images.

It is possible to install the tzdata package and provide the timezone in the .env file. I could make a PR for this if wanted.

Resolving patch:
diff --git a/core/admin/mailu/configuration.py b/core/admin/mailu/configuration.py
--- a/core/admin/mailu/configuration.py
+++ b/core/admin/mailu/configuration.py
@@ -72,6 +72,7 @@
     'SESSION_LIFETIME': 24,
     'SESSION_COOKIE_SECURE': True,
     'CREDENTIAL_ROUNDS': 12,
+    'TZ': 'Etc/UTC',
     # Host settings
     'HOST_IMAP': 'imap',
     'HOST_LMTP': 'imap:2525',


Test patch:
Repository: Mailu/Mailu

Issue:
[master] Typo in front port
In https://github.com/Mailu/Mailu/blob/dbbfa44461c3b8a537e9fe6b56f4690417d3c8a2/core/nginx/conf/nginx.conf#L280 port number must be 10143, not 10043. Just cosmetical, will have no other effect.


Discussion:

Resolving patch:
diff --git a/core/admin/mailu/internal/nginx.py b/core/admin/mailu/internal/nginx.py
--- a/core/admin/mailu/internal/nginx.py
+++ b/core/admin/mailu/internal/nginx.py
@@ -27,12 +27,12 @@
     }),
 }
 
-def check_credentials(user, password, ip, protocol=None):
+def check_credentials(user, password, ip, protocol=None, auth_port=None):
     if not user or not user.enabled or (protocol == "imap" and not user.enable_imap) or (protocol == "pop3" and not user.enable_pop):
         return False
     is_ok = False
     # webmails
-    if len(password) == 64 and ip == app.config['WEBMAIL_ADDRESS']:
+    if len(password) == 64 and auth_port in ['10143', '10025']:
         if user.verify_temp_token(password):
             is_ok = True
     # All tokens are 32 characters hex lowercase
@@ -100,7 +100,7 @@ def handle_authentication(headers):
                 app.logger.warn(f'Invalid user {user_email!r}: {exc}')
             else:
                 ip = urllib.parse.unquote(headers["Client-Ip"])
-                if check_credentials(user, password, ip, protocol):
+                if check_credentials(user, password, ip, protocol, headers["Auth-Port"]):
                     server, port = get_server(headers["Auth-Protocol"], True)
                     return {
                         "Auth-Status": "OK",


Test patch:
Repository: Mailu/Mailu

Issue:
Error 404 not found when opening admin after upgrade 1.8 to master
## Before you open your issue
- [X] Check if no issue or pull-request for this already exists.
- [X] Check [documentation](https://mailu.io/master/) and [FAQ](https://mailu.io/master/faq.html). (Tip, use the search function on the documentation page)
- [X] You understand `Mailu` is made by volunteers in their **free time** — be conscise, civil and accept that delays can occur.
- [X] The title of the issue should be short and simple. It should contain specific terms related to the actual issue. Be specific while writing the title.

## Environment & Versions
### Environment
 - [X] docker-compose

### Versions

Before upgrade: Docker 1.8 images.
After upgrade: Docker master images (pulled 30 December 2021).

## Description

**Mailu 1.8** image redirects `/admin` to `/admin/ui`.

**Mailu master** image no longer redirects `/admin/ui` as the `ui` part in the URL has been removed according to [Tomcat 1929.enhacement](https://github.com/Mailu/Mailu/blob/master/towncrier/newsfragments/1929.enhancement):

> Removed the /admin/ prefix to reduce complexity of routing with Mailu. Admin is accessible directly via /admin instead of /admin/ui

After the upgrade from `1.8` to `master` and visiting the admin page, the browser still uses the cached URL `/admin/ui` and results in 404 not found.


## Replication Steps

1. Create 1.8 production environment on AMD64 platform using `mailu 1.8 Docker images`.
2. Make sure the Admin page works.
3. Remove docker containers (`docker-compose down`).
4. Recreate **all** containers at the same time using `mailu master Docker images`.
5. Open root mail domain. The browser uses the cached URL `admin/ui` and shows Error 404 not found.

Note:  Tested with `TLS_FLAVOR=letsencrypt`, admin and roundcube and Firefox.


## Expected behaviour

Backwards compatibility after Mailu 1.8 upgrade without the need of removing browser caches.

## Front log

```
front_1     | <IP> - - [30/Dec/2021:10:14:35 +0000] "GET /admin/ui/ HTTP/2.0" 404 198 "https://mail.mydomain.nl/sso/login" "Mozilla/5.0 (X11; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0"
```

## Bugfix

Proposal is to redirect `/admin/ui` always to `/admin` to prevent browser caching problems after the upgrade.


Discussion:

Resolving patch:
diff --git a/core/admin/mailu/ui/views/base.py b/core/admin/mailu/ui/views/base.py
--- a/core/admin/mailu/ui/views/base.py
+++ b/core/admin/mailu/ui/views/base.py
@@ -11,6 +11,10 @@
 def index():
     return flask.redirect(flask.url_for('.user_settings'))
 
+@ui.route('/ui/')
+def redirect_old_path():
+    return flask.redirect(flask.url_for('.index'), code=301)
+
 @ui.route('/announcement', methods=['GET', 'POST'])
 @access.global_admin
 def announcement():


Test patch:
Repository: Mailu/Mailu

Issue:
fix key error in dictionary
## What type of PR?

(Feature, enhancement, bug-fix, documentation)

## What does this PR do?

### Related issue(s)
- Auto close an issue like: closes #2145

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.



Discussion:
Thanks for submitting this pull request.
Bors-ng will now build test images. When it succeeds, we will continue to review and test your PR.

bors try

Note: if this build fails, [read this](http://mailu.io/master/contributors/environment.html#when-bors-try-fails).
## try

Build failed:
  * [CI-Done](https://github.com/Mailu/Mailu/runs/4729305720?check_suite_focus=true)

Resolving patch:
diff --git a/core/admin/mailu/internal/views/auth.py b/core/admin/mailu/internal/views/auth.py
--- a/core/admin/mailu/internal/views/auth.py
+++ b/core/admin/mailu/internal/views/auth.py
@@ -18,7 +18,8 @@ def nginx_authentication():
         response.headers['Auth-Error-Code'] = '502 5.5.1'
         utils.limiter.rate_limit_ip(client_ip)
         return response
-    if utils.limiter.should_rate_limit_ip(client_ip):
+    is_from_webmail = headers['Auth-Port'] in ['10143', '10025']
+    if not is_from_webmail and utils.limiter.should_rate_limit_ip(client_ip):
         status, code = nginx.get_status(flask.request.headers['Auth-Protocol'], 'ratelimit')
         response = flask.Response()
         response.headers['Auth-Status'] = status
@@ -31,7 +32,6 @@ def nginx_authentication():
     for key, value in headers.items():
         response.headers[key] = str(value)
     is_valid_user = False
-    is_from_webmail = headers['Auth-Port'] in ['10143', '10025']
     if response.headers.get("Auth-User-Exists"):
         username = response.headers["Auth-User"]
         if utils.limiter.should_rate_limit_user(username, client_ip):


Test patch:
Repository: Mailu/Mailu

Issue:
Admin User Quota sorting is off
Thank you for opening an issue with Mailu. Please understand that issues are meant for bugs and enhancement-requests.
For **user-support questions**, reach out to us  on [matrix](https://matrix.to/#/#mailu:tedomum.net).

To be able to help you best, we need some more information.

## Before you open your issue
- [ x] Check if no issue or pull-request for this already exists.
- [ x] Check [documentation](https://mailu.io/master/) and [FAQ](https://mailu.io/master/faq.html). (Tip, use the search function on the documentation page)
- [ x] You understand `Mailu` is made by volunteers in their **free time** — be conscise, civil and accept that delays can occur.
- [ x] The title of the issue should be short and simple. It should contain specific terms related to the actual issue. Be specific while writing the title.

## Environment & Versions
### Environment
 - [ x] docker-compose
 - [ ] kubernetes
 - [ ] docker swarm

### Versions
1.9

## Description
When sorting by quota in the Admin interface the numbers are sorted like text instead of by number and bytes.


## Expected behaviour
kB is smaller than MB is smaller than GB


![quota](https://user-images.githubusercontent.com/561565/148793187-c7464ef3-b31e-48d5-966f-07d4be315412.PNG)




Discussion:

Resolving patch:
diff --git a/core/admin/mailu/__init__.py b/core/admin/mailu/__init__.py
--- a/core/admin/mailu/__init__.py
+++ b/core/admin/mailu/__init__.py
@@ -57,6 +57,7 @@ def inject_defaults():
         return dict(
             signup_domains= signup_domains,
             config        = app.config,
+            get_locale    = utils.get_locale,
         )
 
     # Jinja filters


Test patch: