Repository: Mailu/Mailu

Issue:
Overriding rspamd forbidden_file_extension.map doesn't work as documented
<!--

Thank you for opening an issue with Mailu. Please understand that issues are meant for bugs and enhancement-requests.
For **user-support questions**, reach out to us  on [matrix](https://matrix.to/#/#mailu:tedomum.net).

To be able to help you best, we need some more information.

Before you open your issue
- Check if no issue or pull-request for this already exists.
- Check [documentation](https://mailu.io/master/) and [FAQ](https://mailu.io/master/faq.html). (Tip, use the search function on the documentation page)
- You understand `Mailu` is made by volunteers in their **free time** — be concise, civil and accept that delays can occur.
- The title of the issue should be short and simple. It should contain specific terms related to the actual issue. Be specific while writing the title.

Please put your text outside of the comment blocks to be visible. You can use the button "Preview" above to check.

-->

## Environment & Version
The host is Ubuntu 22.04
### Environment

- [x] docker compose
- [ ] kubernetes
- [ ] docker swarm

### Version

- Version: `2.0`

<!--
To find your version, get the image name of a mailu container and read  the version from the tag (example for version 1.7).

$> docker ps -a | grep mailu
140b09d4b09c    mailu/roundcube:1.7    "docker-php-entrypoi…"    2 weeks ago    Up 2 days (healthy)    80/tcp
$> grep MAILU_VERSION docker-compose.yml mailu.env
-->

## Description
<!--
Further explain the bug in a few words. It should be clear what the unexpected behaviour is.  Share it in an easy-to-understand language.
-->
My forbidden_file_extension.map placed in `overrides/rspamd/` is ignored and the default file is used instead.

## Replication Steps
<!--
Steps for replicating your issue
-->

With a Mailu 2.0 instance, follow the [instructions](https://mailu.io/2.0/antispam.html#can-i-change-the-list-of-authorized-file-attachments) for changing allowed attachments.  Change `overrides/rspamd/forbidden_file_extension.map` and add/remove some lines, then run `docker compose restart antispam`.  Re-run the `docker compose exec antispam cat /etc/rspamd/local.d/forbidden_file_extension.map` command to see the file used by rspamd.

## Observed behaviour
<!--
Explain or paste the result you received.
-->
I get the output of `core/rspamd/conf/forbidden_file_extension.map` from the Mailu distribution.  Indeed, attachments allowed are still in line with this file, rather than the new changed file.

## Expected behaviour
<!--
Explain what results you expected - be as specific as possible.
Just saying "it doesn’t work as expected" is not useful. It's also helpful to describe what you actually experienced.
-->
I'd expect the modified forbidden_file_extension.map in the `overrides/rspamd` directory is taken into account.  For instance, if I remove js from the list, I should be able to send an attachment with the js extension.

<!--
Often it is very useful to include log fragments of the involved component.
You can get the logs via `docker logs <container name> --tail 1000`.
For example for the admin container: `docker logs mailu_admin_1 --tail 1000`
or using docker compose `docker compose -f /mailu/docker-compose.yml logs --tail 1000 admin`

If you can find the relevant section, please share only the parts that seem relevant. If you have any logs, please enclose them in code tags, like so:

```
Your logs here!
```
-->
## My diagnosis
The overridden file is available at `/overrides/forbidden_file_extension.map` in the antispam container, as expected.  However, `core/rspamd/start.py` adds `forbidden_file_extension.map` to the `config_files` variable so the override file at `/overrides/forbidden_file_extension.map` is ignored and not copied over.  The start script runs Jinja on `/conf/forbidden_file_extension.map` but since there's no directives it's just copied verbatim to `/etc/rspamd/local.d/`.  This default file is what rspamd reads.  The `multimap.conf` file references the map file, but that has no directive for overriding the path of the map file so overriding `multimap.conf` doesn't help.

## Possible solution
To get the expected behaviour (my override file is copied to `/etc/rspamd/local.d/` instead of the default file), I suspect the start script needs to be altered.  I'm unsure how this was intended to be done originally, but I'd probably modify the start script so an administrator can specify whether certain override configuration files should be copied despite being on the `config_files` list.  A pleasant way to still keep Jinja flexibility is for the copied override configuration files to be processed as templates too.  For instance I could copy the `multimap.conf` template to a file in the overrides directory, change it and let the start script process the new template.  This is nicely used in Kolla Ansible for instance.  If it's something relatively straightforward like this I'd be happy to make a PR for it, but might take some time since it's my first issue/PR.


Discussion:
This is indeed a defect. Those instructions will not work. You are correct that original configuration files are not overridden. This is by design. They should not be overwritten. For all *.conf files the rspamd native import mechanism is used. 

`forbidden_file_extension.map` is an exception to the rule that this file must have the possibility to be overridden.
It is sufficient to create an exception for this map in start.py file that it can be overridden.

I will create a fix for this.

Resolving patch:
diff --git a/core/rspamd/start.py b/core/rspamd/start.py
--- a/core/rspamd/start.py
+++ b/core/rspamd/start.py
@@ -16,7 +16,8 @@
 config_files = []
 for rspamd_file in glob.glob("/conf/*"):
     conf.jinja(rspamd_file, env, os.path.join("/etc/rspamd/local.d", os.path.basename(rspamd_file)))
-    config_files.append(os.path.basename(rspamd_file))
+    if rspamd_file != '/conf/forbidden_file_extension.map':
+        config_files.append(os.path.basename(rspamd_file))
 
 for override_file in glob.glob("/overrides/*"):
     if os.path.basename(override_file) not in config_files:


Test patch: