Repository: Mailu/Mailu

Issue:
DNS and Dkim Problem
In the anti-spam logs, I get these messages
`
24.09.2019, 11:18:00 | rspamd_proxy | 6 | proxy | ea17f2 | cannot load dkim key /dkim/gmail.com.dkim.key: cannot stat key file: '/dkim/gmail.com.dkim.key' No such file or directory
-- | -- | -- | -- | -- | --
24.09.2019, 11:13:36 | controller | 8 | map | fdp86m | cannot resolve maps.rspamd.com: query timed out
24.09.2019, 11:07:43 | controller | 8 | map | f61zu5 | cannot resolve maps.rspamd.com: query timed out
24.09.2019, 10:43:39 | rspamd_proxy | 6 | proxy | 6a4515 | cannot load dkim key /dkim/yandex.ru.dkim.key: cannot stat key file: '/dkim/yandex.ru.dkim.key' No such file or directory
24.09.2019, 10:43:01 | rspamd_proxy | 6 | proxy | a2ab83 | cannot load dkim key /dkim/gmail.com.dkim.key: cannot stat key file: '/dkim/gmail.com.dkim.key' No such file or directory
24.09.2019, 10:25:29 | rspamd_proxy | 6 | lua | bd13df | Error querying DNS(79b02fcb6777cde57178ac941273706740344a80.ebl.msbl.org): query timed out
24.09.2019, 10:25:29 | rspamd_proxy | 6 | lua | bd13df | Error querying DNS(fo1wk7518dya4hdfz8g3g7zdfb1c35yw.email.rspamd.com): query timed out
24.09.2019, 10:25:29 | rspamd_proxy | 6 | lua | bd13df | Error querying DNS(qu96fwbu1qxunkig79rpxczqoj7be4ru.email.rspamd.com): query timed out
24.09.2019, 10:25:29 | rspamd_proxy | 6 | lua | bd13df | Error querying DNS(a5f235fe09bd538afb3515a78f2462e510d415af.ebl.msbl.org): query timed out
24.09.2019, 10:16:21 | rspamd_proxy | 6 | lua | aa1a3a | Error querying DNS(fo1wk7518dya4hdfz8g3g7zdfb1c35yw.email.rspamd.com): query timed out
`
What could be the problem?


Discussion:
I have similar problem:
`antispam_1   | 2019-10-27 14:39:37 #10(controller) <zcfokz>; map; rspamd_kv_list_fin: read hash of 1531 elements
antispam_1   | 2019-10-27 14:39:42 #10(controller) <fdp86m>; map; rspamd_map_dns_callback: cannot resolve maps.rspamd.com
antispam_1   | 2019-10-27 14:39:42 #10(controller) <k8f914>; map; rspamd_map_dns_callback: cannot resolve maps.rspamd.com
antispam_1   | 2019-10-27 14:39:42 #10(controller) <k8f914>; map; rspamd_map_dns_callback: cannot resolve maps.rspamd.com
antispam_1   | 2019-10-27 14:39:42 #10(controller) <fdp86m>; map; rspamd_map_dns_callback: cannot resolve maps.rspamd.com
antispam_1   | 2019-10-27 14:39:42 #10(controller) <xa6qqy>; map; rspamd_map_dns_callback: cannot resolve rspamd.com
antispam_1   | 2019-10-27 14:39:42 #10(controller) <pr5857>; map; rspamd_map_dns_callback: cannot resolve updates.rspamd.com
antispam_1   | 2019-10-27 14:39:42 #10(controller) <fdp86m>; map; rspamd_map_dns_callback: cannot resolve maps.rspamd.com`

I run iptables -t nat -A POSTROUTING ! -o docker0 -s 192.168.203.0/24 -j MASQUERADE
and now it is working.
Many of these problems are indeed related to people having way too general outbound masquerading rules.

You should never masquerade internal traffic to your Docker. Please use specific masquerading rules. Also, blogwebpl's suggestion will not hold since compose projects will have their own host interface instead of docker0.
Hi! I think you answered to the DNS errors problem, but there is still the issue with the dkim keys which cannot be loaded. I have the exact same issue:

    cannot load dkim key /dkim/gmail.com.dkim.key: cannot stat key file: '/dkim/gmail.com.dkim.key' No such file or directory

It seems pretty normal for rspamd to not be able to find these keys since there are for external domains. It looks identical to [this issue](https://github.com/rspamd/rspamd/issues/2832), so I bet for a configuration issue ([here?](https://github.com/Mailu/Mailu/blob/master/core/rspamd/conf/dkim_signing.conf#L1)). I'm not sure about the reason to use fallback, and even less about the implication of keeping it as it is or not, so I'll let you answer that :wink: 
Thanks for insisting on the issue about falling back. This might be an easy fix indeed.
Hi There,

The `Mailu`-Project is currently in a bit of a bind! We are short on man-power, and we need to judge if it is possible for us to put in some work on this issue.

To help with that, we are currently trying to find out which issues are actively keeping users from using `Mailu`, which issues have someone who want to work on them — and which issues may be less important. These a less important ones could be discarded for the time being, until the project is in a more stable and regular state once again.

In order for us to better assess this, it would be helpful if you could put a **reaction on this post** (use the :smiley: icon to the top-right).
- 👍️ if you **need this** to be able to use Mailu. Ideally, you’d also be able to test this on your installation, and provide feedback …
- 🎉 if you find it a **nice bonus**, but no deal-breaker
- 🚀 if you want to **work on it yourself**!
We want to keep this **voting open for 2 weeks** from now, so please help out!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Ran into this issue, my fix for the DKIM problem was to generate the Rspamd dkim key file.

`docker exec -it mailu_antispam_1 /bin/bash
rspamadm dkim_keygen -s 'dkim' -d example.com -k /dkim/example.com.dkim.key`
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Resolving patch:
diff --git a/core/admin/mailu/internal/views/__init__.py b/core/admin/mailu/internal/views/__init__.py
--- a/core/admin/mailu/internal/views/__init__.py
+++ b/core/admin/mailu/internal/views/__init__.py
@@ -1,3 +1,3 @@
 __all__ = [
-    'auth', 'postfix', 'dovecot', 'fetch'
+    'auth', 'postfix', 'dovecot', 'fetch', 'rspamd'
 ]
diff --git a/core/admin/mailu/internal/views/rspamd.py b/core/admin/mailu/internal/views/rspamd.py
new file mode 100644
--- /dev/null
+++ b/core/admin/mailu/internal/views/rspamd.py
@@ -0,0 +1,30 @@
+from mailu import models
+from mailu.internal import internal
+
+import flask
+
+def vault_error(*messages, status=404):
+    return flask.make_response(flask.jsonify({'errors':messages}), status)
+
+# rspamd key format:
+# {"selectors":[{"pubkey":"...","domain":"...","valid_start":TS,"valid_end":TS,"key":"...","selector":"...","bits":...,"alg":"..."}]}
+
+# hashicorp vault answer format:
+# {"request_id":"...","lease_id":"","renewable":false,"lease_duration":2764800,"data":{...see above...},"wrap_info":null,"warnings":null,"auth":null}
+
+@internal.route("/rspamd/vault/v1/dkim/<domain_name>", methods=['GET'])
+def rspamd_dkim_key(domain_name):
+    domain = models.Domain.query.get(domain_name) or flask.abort(vault_error('unknown domain'))
+    key = domain.dkim_key or flask.abort(vault_error('no dkim key', status=400))
+    return flask.jsonify({
+        'data': {
+            'selectors': [
+                {
+                    'domain'  : domain.name,
+                    'key'     : key.decode('utf8'),
+                    'selector': flask.current_app.config.get('DKIM_SELECTOR', 'dkim'),
+                }
+            ]
+        }
+    })
+
diff --git a/core/rspamd/start.py b/core/rspamd/start.py
--- a/core/rspamd/start.py
+++ b/core/rspamd/start.py
@@ -11,6 +11,7 @@
 # Actual startup script
 
 os.environ["REDIS_ADDRESS"] = system.get_host_address_from_environment("REDIS", "redis")
+os.environ["ADMIN_ADDRESS"] = system.get_host_address_from_environment("ADMIN", "admin")
 
 if os.environ.get("ANTIVIRUS") == 'clamav':
     os.environ["ANTIVIRUS_ADDRESS"] = system.get_host_address_from_environment("ANTIVIRUS", "antivirus:3310")


Test patch: