Repository: Mailu/Mailu

Issue:
Rainloop: Missing support for '*_ADDRESS' env vars
## Environment & Versions
### Environment
`docker-compose`

### Versions
`1.9`

## Description
Rainloop's `start.py` calls `system.resolve_address()` instead of `system.get_host_address_from_environment()`, thus name resolution breaks when using `*_ADDRESS` env vars.

As a workaround one can set `HOST_FRONT` and `HOST_IMAP` manually on the `rainloop` container to get the same behaviour, but it would be great to use the existing `mailu.env` for `rainloop` too.

## Expected behaviour
`rainloop` honors `FRONT_ADDRESS` and `IMAP_ADDRESS`, when set in `mailu.env`.
Rainloop: Missing support for '*_ADDRESS' env vars
## Environment & Versions
### Environment
`docker-compose`

### Versions
`1.9`

## Description
Rainloop's `start.py` calls `system.resolve_address()` instead of `system.get_host_address_from_environment()`, thus name resolution breaks when using `*_ADDRESS` env vars.

As a workaround one can set `HOST_FRONT` and `HOST_IMAP` manually on the `rainloop` container to get the same behaviour, but it would be great to use the existing `mailu.env` for `rainloop` too.

## Expected behaviour
`rainloop` honors `FRONT_ADDRESS` and `IMAP_ADDRESS`, when set in `mailu.env`.
Autotest for Rainloop admin console
In relation to #947, we should create a small script in the test suite which checks if the Rainloop admin interface is indeed disabled by default. Just to be safe for any future changes in config parsing of Rainloop.

A simple implementation would be testing the HTTP return code, however Rainloop still returns 200, OK even when the page is prompting "Access Denied". See RainLoop/rainloop-webmail#1838. If that gets not solved, we might resort to parsing the result page. THat would be more error prone if they ever choose to change the layout of the error page.


Discussion:


Hi There,

The `Mailu`-Project is currently in a bit of a bind! We are short on man-power, and we need to judge if it is possible for us to put in some work on this issue.

To help with that, we are currently trying to find out which issues are actively keeping users from using `Mailu`, which issues have someone who want to work on them — and which issues may be less important. These a less important ones could be discarded for the time being, until the project is in a more stable and regular state once again.

In order for us to better assess this, it would be helpful if you could put a **reaction on this post** (use the :smiley: icon to the top-right).
- 👍️ if you **need this** to be able to use Mailu. Ideally, you’d also be able to test this on your installation, and provide feedback …
- 🎉 if you find it a **nice bonus**, but no deal-breaker
- 🚀 if you want to **work on it yourself**!
We want to keep this **voting open for 2 weeks** from now, so please help out!
I've opened a rainloop PR for this:
https://github.com/RainLoop/rainloop-webmail/pull/2029

Resolving patch:
diff --git a/webmails/snappymail/config.py b/webmails/snappymail/config.py
deleted file mode 100755
--- a/webmails/snappymail/config.py
+++ /dev/null
@@ -1,16 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import logging as log
-import sys
-
-from socrate import system, conf
-
-args = os.environ.copy()
-
-log.basicConfig(stream=sys.stderr, level=args.get("LOG_LEVEL", "WARNING"))
-
-# Build final configuration paths
-conf.jinja("/config/nginx-snappymail.conf", args, "/etc/nginx/http.d/snappymail.conf")
-if os.path.exists("/var/run/nginx.pid"):
-    os.system("nginx -s reload")
diff --git a/webmails/snappymail/start.py b/webmails/snappymail/start.py
deleted file mode 100755
--- a/webmails/snappymail/start.py
+++ /dev/null
@@ -1,34 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import shutil
-import logging as log
-import sys
-import subprocess
-
-from socrate import system, conf
-
-log.basicConfig(stream=sys.stderr, level=os.environ.get("LOG_LEVEL", "WARNING"))
-
-# Actual startup script
-os.environ["FRONT_ADDRESS"] = system.resolve_address(os.environ.get("HOST_FRONT", "front"))
-os.environ["IMAP_ADDRESS"] = system.resolve_address(os.environ.get("HOST_IMAP", "imap"))
-
-os.environ["MAX_FILESIZE"] = str(int(int(os.environ.get("MESSAGE_SIZE_LIMIT"))*0.66/1048576))
-
-base = "/data/_data_/_default_/"
-shutil.rmtree(base + "domains/", ignore_errors=True)
-os.makedirs(base + "domains", exist_ok=True)
-os.makedirs(base + "configs", exist_ok=True)
-
-conf.jinja("/defaults/default.ini", os.environ, "/data/_data_/_default_/domains/default.ini")
-conf.jinja("/defaults/application.ini", os.environ, "/data/_data_/_default_/configs/application.ini")
-conf.jinja("/defaults/php.ini", os.environ, "/etc/php81/php.ini")
-# Start the fastcgi process manager now that config files have been adjusted
-os.system("php-fpm81")
-
-os.system("chown -R nginx:nginx /data")
-os.system("chmod -R a+rX /var/www/webmail/")
-
-subprocess.call(["/config.py"])
-os.execv("/usr/sbin/nginx", ["nginx", "-g", "daemon off;"])
diff --git a/webmails/roundcube/start.py b/webmails/start.py
similarity index 72%
rename from webmails/roundcube/start.py
rename to webmails/start.py
--- a/webmails/roundcube/start.py
+++ b/webmails/start.py
@@ -4,9 +4,10 @@
 import logging
 import sys
 import subprocess
+import shutil
 import hmac
 
-from socrate import conf
+from socrate import conf, system
 
 env = os.environ
 
@@ -17,6 +18,8 @@
 context.update(env)
 
 context["MAX_FILESIZE"] = str(int(int(env.get("MESSAGE_SIZE_LIMIT", "50000000")) * 0.66 / 1048576))
+context["FRONT_ADDRESS"] = system.get_host_address_from_environment("FRONT", "front")
+context["IMAP_ADDRESS"] = system.get_host_address_from_environment("IMAP", "imap")
 
 db_flavor = env.get("ROUNDCUBE_DB_FLAVOR", "sqlite")
 if db_flavor == "sqlite":
@@ -52,7 +55,7 @@
 
 # roundcube plugins
 # (using "dict" because it is ordered and "set" is not)
-plugins = dict((p, None) for p in env.get("ROUNDCUBE_PLUGINS", "").replace(" ", "").split(",") if p and os.path.isdir(os.path.join("/var/www/webmail/plugins", p)))
+plugins = dict((p, None) for p in env.get("ROUNDCUBE_PLUGINS", "").replace(" ", "").split(",") if p and os.path.isdir(os.path.join("/var/www/roundcube/plugins", p)))
 if plugins:
     plugins["mailu"] = None
 else:
@@ -67,15 +70,14 @@
 context["SESSION_TIMEOUT_MINUTES"] = max(int(env.get("SESSION_TIMEOUT", "3600")) // 60, 1)
 
 # create config files
-conf.jinja("/conf/php.ini", context, "/etc/php81/php.ini")
-conf.jinja("/conf/config.inc.php", context, "/var/www/webmail/config/config.inc.php")
+conf.jinja("/conf/config.inc.php", context, "/var/www/roundcube/config/config.inc.php")
 
 # create dirs
 os.system("mkdir -p /data/gpg")
 
 print("Initializing database")
 try:
-    result = subprocess.check_output(["/var/www/webmail/bin/initdb.sh", "--dir", "/var/www/webmail/SQL"],
+    result = subprocess.check_output(["/var/www/roundcube/bin/initdb.sh", "--dir", "/var/www/roundcube/SQL"],
                                      stderr=subprocess.STDOUT)
     print(result.decode())
 except subprocess.CalledProcessError as exc:
@@ -88,22 +90,30 @@
 
 print("Upgrading database")
 try:
-    subprocess.check_call(["/var/www/webmail/bin/update.sh", "--version=?", "-y"], stderr=subprocess.STDOUT)
+    subprocess.check_call(["/var/www/roundcube/bin/update.sh", "--version=?", "-y"], stderr=subprocess.STDOUT)
 except subprocess.CalledProcessError as exc:
     exit(4)
 else:
     print("Cleaning database")
     try:
-        subprocess.check_call(["/var/www/webmail/bin/cleandb.sh"], stderr=subprocess.STDOUT)
+        subprocess.check_call(["/var/www/roundcube/bin/cleandb.sh"], stderr=subprocess.STDOUT)
     except subprocess.CalledProcessError as exc:
         exit(5)
 
+base = "/data/_data_/_default_/"
+shutil.rmtree(base + "domains/", ignore_errors=True)
+os.makedirs(base + "domains", exist_ok=True)
+os.makedirs(base + "configs", exist_ok=True)
+
+conf.jinja("/defaults/default.json", context, "/data/_data_/_default_/domains/default.json")
+conf.jinja("/defaults/application.ini", context, "/data/_data_/_default_/configs/application.ini")
+conf.jinja("/defaults/php.ini", context, "/etc/php81/php.ini")
+
 # setup permissions
-os.system("chown -R nginx:nginx /data")
-os.system("chmod -R a+rX /var/www/webmail/")
+os.system("chown -R mailu:mailu /data")
 
 # Configure nginx
-conf.jinja("/conf/nginx-roundcube.conf", context, "/etc/nginx/http.d/roundcube.conf")
+conf.jinja("/conf/nginx-webmail.conf", context, "/etc/nginx/http.d/webmail.conf")
 if os.path.exists("/var/run/nginx.pid"):
     os.system("nginx -s reload")
 


Test patch:
diff --git a/.github/workflows/build_test_deploy.yml b/.github/workflows/build_test_deploy.yml
--- a/.github/workflows/build_test_deploy.yml
+++ b/.github/workflows/build_test_deploy.yml
@@ -340,7 +340,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        target: ["core", "fetchmail", "filters", "snappymail", "roundcube", "webdav"]
+        target: ["core", "fetchmail", "filters", "webmail", "webdav"]
         time: ["2"]
         include:
           - target: "filters"
@@ -394,7 +394,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        target: ["setup", "docs", "fetchmail", "roundcube", "admin", "traefik-certdumper", "radicale", "clamav", "rspamd", "postfix", "dovecot", "unbound", "nginx", "snappymail"]
+        target: ["setup", "docs", "fetchmail", "webmail", "admin", "traefik-certdumper", "radicale", "clamav", "rspamd", "postfix", "dovecot", "unbound", "nginx"]
     steps:
       - uses: actions/checkout@v3
       - name: Retrieve global variables
@@ -439,7 +439,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        target: ["setup", "docs", "fetchmail", "roundcube", "admin", "traefik-certdumper", "radicale", "clamav", "rspamd", "postfix", "dovecot", "unbound", "nginx", "snappymail"]
+        target: ["setup", "docs", "fetchmail", "webmail", "admin", "traefik-certdumper", "radicale", "clamav", "rspamd", "postfix", "dovecot", "unbound", "nginx"]
     steps:
       - uses: actions/checkout@v3
       - name: Retrieve global variables
diff --git a/tests/build.hcl b/tests/build.hcl
--- a/tests/build.hcl
+++ b/tests/build.hcl
@@ -36,8 +36,7 @@ group "default" {
     "imap",
     "smtp",
 
-    "snappymail",
-    "roundcube",
+    "webmail",
 
     "antivirus",
     "fetchmail",
@@ -169,24 +168,15 @@ target "smtp" {
 }
 
 # -----------------------------------------------------------------------------------------
-# Webmail images
+# Webmail image
 # -----------------------------------------------------------------------------------------
-target "snappymail" {
+target "webmail" {
   inherits = ["defaults"]
-  context = "webmails/snappymail/"
+  context = "webmails/"
   contexts = {
     base = "target:base"
   }
-  tags = tag("snappymail")
-}
-
-target "roundcube" {
-  inherits = ["defaults"]
-  context = "webmails/roundcube/"
-  contexts = {
-    base = "target:base"
-  }
-  tags = tag("roundcube")
+  tags = tag("webmail")
 }
 
 # -----------------------------------------------------------------------------------------
diff --git a/tests/compose/snappymail/docker-compose.yml b/tests/compose/snappymail/docker-compose.yml
deleted file mode 100644
--- a/tests/compose/snappymail/docker-compose.yml
+++ /dev/null
@@ -1,106 +0,0 @@
-# This file is auto-generated by the Mailu configuration wizard.
-# Please read the documentation before attempting any change.
-# Generated for compose flavor
-
-version: '3.6'
-
-services:
-
-  # External dependencies
-  redis:
-    image: redis:alpine
-    restart: always
-    volumes:
-      - "/mailu/redis:/data"
-
-  # Core services
-  front:
-    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}nginx:${MAILU_VERSION:-local}
-    restart: always
-    env_file: mailu.env
-    logging:
-      driver: json-file
-    ports:
-      - "127.0.0.1:80:80"
-      - "127.0.0.1:443:443"
-      - "127.0.0.1:25:25"
-      - "127.0.0.1:465:465"
-      - "127.0.0.1:587:587"
-      - "127.0.0.1:110:110"
-      - "127.0.0.1:995:995"
-      - "127.0.0.1:143:143"
-      - "127.0.0.1:993:993"
-    volumes:
-      - "/mailu/certs:/certs"
-
-  admin:
-    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}admin:${MAILU_VERSION:-local}
-    restart: always
-    env_file: mailu.env
-    volumes:
-      - "/mailu/data:/data"
-      - "/mailu/dkim:/dkim"
-    depends_on:
-      - redis
-      - resolver
-    dns:
-      - 192.168.203.254
-
-  imap:
-    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}dovecot:${MAILU_VERSION:-local}
-    restart: always
-    env_file: mailu.env
-    volumes:
-      - "/mailu/mail:/mail"
-      - "/mailu/overrides:/overrides"
-    depends_on:
-      - front
-
-  smtp:
-    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}postfix:${MAILU_VERSION:-local}
-    restart: always
-    env_file: mailu.env
-    volumes:
-      - "/mailu/overrides:/overrides"
-    depends_on:
-      - front
-
-  antispam:
-    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}rspamd:${MAILU_VERSION:-local}
-    restart: always
-    env_file: mailu.env
-    volumes:
-      - "/mailu/filter:/var/lib/rspamd"
-      - "/mailu/dkim:/dkim"
-      - "/mailu/overrides/rspamd:/etc/rspamd/override.d"
-    depends_on:
-      - front
-
-  # Optional services
-
-  resolver:
-    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}unbound:${MAILU_VERSION:-local}
-    env_file: mailu.env
-    restart: always
-    networks:
-      default:
-        ipv4_address: 192.168.203.254
-
-  # Webmail
-  webmail:
-    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}snappymail:${MAILU_VERSION:-local}
-    restart: always
-    env_file: mailu.env
-    volumes:
-      - "/mailu/webmail:/data"
-    depends_on:
-      - imap
-
-
-networks:
-  default:
-    driver: bridge
-    ipam:
-      driver: default
-      config:
-        - subnet: 192.168.203.0/24
diff --git a/tests/compose/snappymail/mailu.env b/tests/compose/snappymail/mailu.env
deleted file mode 100644
--- a/tests/compose/snappymail/mailu.env
+++ /dev/null
@@ -1,138 +0,0 @@
-# Mailu main configuration file
-#
-# Generated for compose flavor
-#
-# This file is autogenerated by the configuration management wizard.
-# For a detailed list of configuration variables, see the documentation at
-# https://mailu.io
-
-###################################
-# Common configuration variables
-###################################
-
-# Set this to the path where Mailu data and configuration is stored
-# This variable is now set directly in `docker-compose.yml by the setup utility
-# ROOT=/mailu
-
-# Mailu version to run (1.0, 1.1, etc. or master)
-#VERSION=master
-
-# Set to a randomly generated 16 bytes string
-SECRET_KEY=V5J4SHRYVW9PZIQU
-
-# Address where listening ports should bind
-# This variables are now set directly in `docker-compose.yml by the setup utility
-# PUBLIC_IPV4= 127.0.0.1 (default: 127.0.0.1)
-# PUBLIC_IPV6=  (default: ::1)
-
-# Subnet of the docker network. This should not conflict with any networks to which your system is connected. (Internal and external!)
-SUBNET=192.168.203.0/24
-
-# Main mail domain
-DOMAIN=mailu.io
-
-# Hostnames for this server, separated with comas
-HOSTNAMES=localhost
-
-# Postmaster local part (will append the main mail domain)
-POSTMASTER=admin
-
-# Choose how secure connections will behave (value: letsencrypt, cert, notls, mail, mail-letsencrypt)
-TLS_FLAVOR=cert
-
-# Authentication rate limit (per source IP address)
-AUTH_RATELIMIT=10/minute;1000/hour 
-
-# Opt-out of statistics, replace with "True" to opt out
-DISABLE_STATISTICS=False
-
-###################################
-# Optional features
-###################################
-
-# Expose the admin interface (value: true, false)
-ADMIN=false
-
-# Choose which webmail to run if any (values: roundcube, snappymail, none)
-WEBMAIL=snappymail
-
-# Dav server implementation (value: radicale, none)
-WEBDAV=none
-
-# Antivirus solution (value: clamav, none)
-#ANTIVIRUS=none
-
-#Antispam solution
-ANTISPAM=none
-
-###################################
-# Mail settings
-###################################
-
-# Message size limit in bytes
-# Default: accept messages up to 50MB
-MESSAGE_SIZE_LIMIT=50000000
-
-# Networks granted relay permissions
-# Use this with care, all hosts in this networks will be able to send mail without authentication!
-RELAYNETS=
-
-# Will relay all outgoing mails if configured
-RELAYHOST=
-
-# Fetchmail delay
-FETCHMAIL_DELAY=600
-
-# Recipient delimiter, character used to delimiter localpart from custom address part
-RECIPIENT_DELIMITER=+
-
-# DMARC rua and ruf email
-DMARC_RUA=admin
-DMARC_RUF=admin
-
-
-# Maildir Compression
-# choose compression-method, default: none (value: gz, bz2, lz4, zstd)
-COMPRESSION=
-# change compression-level, default: 6 (value: 1-9)
-COMPRESSION_LEVEL=
-
-###################################
-# Web settings
-###################################
-
-# Path to the admin interface if enabled
-WEB_ADMIN=/admin
-
-# Path to the webmail if enabled
-WEB_WEBMAIL=/webmail
-
-# Website name
-SITENAME=Mailu
-
-# Linked Website URL
-WEBSITE=https://mailu.io
-
-
-
-###################################
-# Advanced settings
-###################################
-
-# Log driver for front service. Possible values:
-# json-file (default)
-# journald (On systemd platforms, useful for Fail2Ban integration)
-# syslog (Non systemd platforms, Fail2Ban integration. Disables `docker-compose log` for front!)
-# LOG_DRIVER=json-file
-
-# Docker-compose project name, this will prepended to containers names.
-COMPOSE_PROJECT_NAME=mailu
-
-# Header to take the real ip from
-REAL_IP_HEADER=
-
-# IPs for nginx set_real_ip_from (CIDR list separated by commas)
-REAL_IP_FROM=
-
-# choose wether mailu bounces (no) or rejects (yes) mail when recipient is unknown (value: yes, no)
-REJECT_UNLISTED_RECIPIENT=
diff --git a/tests/compose/webmail/01_ensure_admin_unreachable.sh b/tests/compose/webmail/01_ensure_admin_unreachable.sh
new file mode 100755
--- /dev/null
+++ b/tests/compose/webmail/01_ensure_admin_unreachable.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+
+IP="$(docker inspect webmail_webmail_1|jq -r '.[0].NetworkSettings.Networks.webmail_default.IPAddress')"
+
+MAIN_RETURN_CODE=$(curl -I -so /dev/null -w "%{http_code}" http://$IP/)
+[[ $MAIN_RETURN_CODE -ne 200 && $MAIN_RETURN_CODE -ne 302 ]] && echo "The default page of snappymail hasn't returned 200 but $MAIN_RETURN_CODE!" >>/dev/stderr && exit 1
+[[ $(curl -I -so /dev/null -w "%{http_code}" http://$IP/?admin) -ne 403 ]] && echo "The admin of snappymail is not disabled!" >>/dev/stderr && exit 1
+echo "Everything OK" >/dev/stderr
+
+exit 0
diff --git a/tests/compose/roundcube/docker-compose.yml b/tests/compose/webmail/docker-compose.yml
similarity index 96%
rename from tests/compose/roundcube/docker-compose.yml
rename to tests/compose/webmail/docker-compose.yml
--- a/tests/compose/roundcube/docker-compose.yml
+++ b/tests/compose/webmail/docker-compose.yml
@@ -88,7 +88,7 @@ services:
 
   # Webmail
   webmail:
-    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}roundcube:${MAILU_VERSION:-local}
+    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}webmail:${MAILU_VERSION:-local}
     restart: always
     env_file: mailu.env
     volumes:
diff --git a/tests/compose/roundcube/mailu.env b/tests/compose/webmail/mailu.env
similarity index 99%
rename from tests/compose/roundcube/mailu.env
rename to tests/compose/webmail/mailu.env
--- a/tests/compose/roundcube/mailu.env
+++ b/tests/compose/webmail/mailu.env
@@ -54,7 +54,7 @@ DISABLE_STATISTICS=False
 ADMIN=false
 
 # Choose which webmail to run if any (values: roundcube, snappymail, none)
-WEBMAIL=roundcube
+WEBMAIL=snappymail
 
 # Dav server implementation (value: radicale, none)
 WEBDAV=none