Repository: Mailu/Mailu

Issue:
Brainstorming for 1.10 / 2.0 roadmap
# Description
Now 1.9 has been released for some time, it is time to discuss what we want to work on for the next release.

- Release date?
- Scope 1.10 (normal release) or 2.0 (release with breaking changes)?  
- What do we want to work on?

## Ideas

Ideas from the last meeting:
- introduce snappymail.
- rootless containers
  - probably requires init container
- replacement of Socrates. ghostwheel42 will work on this (if he finds time).
- A security/keys page for Mailu admin interface. 
  - On this page you can configure/change/generate keys for all your Mail domains.
  - DNS overview 
    - Have some UI that tells you if DNS is correctly configured.
- Have a look at BuildX
  - means we can build for multiple platforms including ARM
  - means we remove the build arm script
  - means CI will be changed massively.
  - For ARM, we could maybe build once per week to make sure the build time for normal builts is not too long.
- autoconfiguration for email clients?
  - automx: https://rseichter.github.io/automx2/?
  - integrate whatever we choose with mailu (part of mailu)?
  - also good time to drop starttls for imap/pop/smtp(sending mail).
    - So only support direct SSL/TLS
    - Could be done via environment_variable. When not configured, then starttls supported. If configured (default for new deployments) then it is disabled.
    - Another idea is to disable starttls, and report a custom error (when you use explicit starttls) that you must switch to implicit SSL/TLS port 465.
- Full 2 factor authentication with xoath2 
  -  Too large in scope for this release. But preparations could be made.
  - Means we need autoconfiguration. Otherwise the email client will not use xoath2.
  - means using single sign on via identity provider (which mailu could be as well). This opens the door to use other identity providers in the future.

Feel free to suggest your own ideas

## Misc
For small non-critical issues/features I suggest we do not put it on the roadmap, but simply offer anyone the chance to pick these up and submit a PR if they want it included.

What are your thoughts? Please share your feedback.
Regardless the above wall of text, feel free to mention any feature/issue you would like included in the next release.

## My 2 cents
For release date we could maybe aim around August/September?
We are all very busy with our personal life now. This should give enough time to implement new features. Of course a decided date is not set in stone. It could still be moved forward/backwards. 

https://github.com/Mailu/Mailu/blob/master/design/mailu-directory-structure.md
This is what I was thinking about for the scope. Changing the directory structure must be part of a major release with breaking changes. Do we want to make this change for this release or postpone this?

Personally I'd like to check 
 - switching to SnappyMail 
 - rootless containers
 - BuildX. At least investigate what changes are required.

Feel free to suggest your own ideas. 



Discussion:
thanks for this great project

just some thoughts here
- rootless containers would be great
- for my personal setup i integrated mailman3 with "official" docker container from "maxking". Was not complicated as an additional/optional setup via ansible within the same docker-compose file/network. Only thing not working here is dkim key generation as i use an additional domain for the mailing lists (e.g. "example.com" for mailu and "lists.example.com" for mailman).
- allow dkim key creation/full listing via "rspamadm vault" (will also solve dkim problem for my mailman setup above)
- maybe more customization beside log and color of upper left box - e.g. the primary/secondary color for lines/buttons and so one
- allow redirect url after admin login (simply setup of additional admin uis and use your existing nginx auth config for that - for me it would help to protect parallel mailgraph installation for mailu and allow easy auth for admins)
- more general: some statistics about postfix, e.g. how much of my mails are rejected from others like google&co as possible spam

and maybe an alternative for automx2 might be email-autoconf (https://gitlab.com/onlime/email-autoconf/-/blob/main/README.md). But have not tested it, just thought about adding this feature to my setup too and stumbled accross these two projects.
My ideas as a first time user of mailu:
 - Having a `port set` for not SSL/TLS use case (local network) like for the webmail but without custom login throttling rules. (useful for external reverse proxy that handle SSL termination but SSL in enabled on `front`)
 - `API` for password reset, user registration, Avatar upload/support.
 - Integration of `caldav` account generation on registration via admin.
 - Standardize `ENV` configration variables.
 - Better documentation about `DOMAIN` and `HOSTNAMES`.
 - Better error handling on startup. During configuration of `init.py`, the python script fail and print error of the key missing, effected config file/ line number (Now it fail and you dont know why. You have to kill the process to see the output stacktrace in logs.)
 - Default `ENV` values on configuration scripts. some scripts dont have default values. (`RECIPIENT_DELIMITER`, `POSTMASTER`, `MESSAGE_SIZE_LIMIT`,`MESSAGE_RATELIMIT` are some)
 - Prevent override of config files if path is in a mounted volume it might have been customized. (Like for the rainloop webmail override didnt work for me)
 - `front`, `admin`, `antispam`, `unbound` containers health check output is huge. a simple 200 / OK would be awesome. (it breaks portainer UI)
 - Automatic detection of stack configuration via PINGs for `WEBMAIL`, `CLAMAV`, `WEBDAV` or get it from an `API@admin`.
 - Possibility to disable reverse proxy rules without disabling the relative service. (Like having `WEBMAIL=rainloop` but nginx configuration block is not set)
 
More advance stuff that mailcow have that to me they seem interesting / useful:
 - Full text search
 - [oletools](https://github.com/decalage2/oletools) integration in `rspamd`
 
 
 
hi

just came around and while testing mailu i came along the same question...

> * more general: some statistics about postfix, e.g. how much of my mails are rejected from others like google&co as possible spam

maybe "integrate" https://gitlab.com/lightmeter/controlcenter to get visual insight into the postfix logs...
Hello, i'm curious to know if it is currently possible to run Mailu using docker in rootless configuration.

Thank you if you could clarify about this topic.
From security point of view rootless containers are a must have, also I'm a bit concerned that all the environment variables are global in default `mailu.env` generated, so all applications have access to all secrets even when they don't need to.

As discussed in https://github.com/Mailu/Mailu/issues/2258 having UTF-8 Dovecot config by default (rather than default strange UTF-7 IMAP) would be nice to.
@nazar-pc most containers now drop privileges early on master and the environment is sanitized so not all secrets are available to everyone. See https://github.com/Mailu/Mailu/blob/5b4f2fb0753709c94563c01e4870c6059b4c7231/core/base/libs/socrate/socrate/system.py#L27

Going rootless is non-trivial but patches would be welcome.
I'd also like to suggest another major breaking change that is `:LAYOUT=fs` for Dovecot. It makes it nicer to manage folders and allows to use somewhat popular `.` character in names of directories. But those who upgrade would need to go through a migration process (or this might just be added to the docs somewhere as it works great with an override).
Please create new tickets/feature requests for all of this so that it can be discussed.

IMHO there is value in sticking with standard Maildir++ format: that's a lot easier to re-use for users migrating away from Mailu ... or those using 3rd party tools on the backend.
One feature that would be awesome to see on the roadmap as well is the configuration of telemetry data and metrics for mailu and its components (I believe https://github.com/Mailu/Mailu/issues/1059 was supposed to cover this).

Dovecot: native, https://doc.dovecot.org/configuration_manual/stats/openmetrics/
Rspamd: native, https://www.rspamd.com/doc/modules/metric_exporter.html
Postfix: e.g. via https://github.com/google/mtail/blob/main/examples/postfix.mtail
Nginx: e.g. via https://github.com/nginxinc/nginx-prometheus-exporter
Flask: e.g. via https://pypi.org/project/prometheus-flask-exporter/
Proposal: mass import tool for users and alias #2711
https://github.com/Mailu/Mailu/issues/2711

Proposal: Test for correct diagnostics configuration #2710
https://github.com/Mailu/Mailu/issues/2710

Resolving patch:
diff --git a/core/admin/mailu/configuration.py b/core/admin/mailu/configuration.py
--- a/core/admin/mailu/configuration.py
+++ b/core/admin/mailu/configuration.py
@@ -31,7 +31,7 @@
     'SQLALCHEMY_TRACK_MODIFICATIONS': False,
     # Statistics management
     'INSTANCE_ID_PATH': '/data/instance',
-    'STATS_ENDPOINT': '19.{}.stats.mailu.io',
+    'STATS_ENDPOINT': '20.{}.stats.mailu.io',
     # Common configuration variables
     'SECRET_KEY': 'changeMe',
     'DOMAIN': 'mailu.io',
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -9,7 +9,7 @@
 source_suffix = '.rst'
 master_doc = 'index'
 project = 'Mailu'
-copyright = '2018, Mailu authors'
+copyright = '2016, Mailu authors'
 author = 'Mailu authors'
 version = release = os.environ.get('VERSION', 'master')
 language = 'en'
@@ -25,7 +25,7 @@
 # to template names.
 html_sidebars = {
     '**': [
-        'relations.html', 
+        'relations.html',
         'searchbox.html',
     ]
 }
@@ -36,10 +36,10 @@
     'github_user': 'mailu',
     'github_repo': 'mailu',
     'github_version': version,
-    'stable_version': '1.9',
+    'stable_version': '2.0',
     'versions': [
-        ('1.8', '/1.8/'),
         ('1.9', '/1.9/'),
+        ('2.0', '/2.0/'),
         ('master', '/master/')
     ],
     'conf_py_path': '/docs/'


Test patch: