Repository: Mailu/Mailu

Issue:
connect to /tmp/podop.socket: Permission denied
Hi,
I build the docker images from Mailu/tests/build_arm.sh (commit 5d20f28).
Setting mailu.env log level to: `LOG_LEVEL=DEBUG`
I got this "Temporary lookup failure" when sending a test email to my domain.
Digging out the postfix's log is the best I can manage with my limited knowledge.
Any help please?
Thanks,
Tin.

`smtp         | Dec 25 21:35:01 mail postfix/trivial-rewrite[276]: warning: connect to /tmp/podop.socket: Permission denied`
`smtp         | Dec 25 21:35:01 mail postfix/trivial-rewrite[276]: warning: table socketmap:unix:/tmp/podop.socket:transport lookup error: Permission denied`
`smtp         | Dec 25 21:35:01 mail postfix/trivial-rewrite[276]: warning: socketmap:unix:/tmp/podop.socket:transport lookup error for "*"`
`smtp         | Dec 25 21:35:01 mail postfix/trivial-rewrite[276]: warning: connect to /tmp/podop.socket: Permission denied`
`smtp         | Dec 25 21:35:01 mail postfix/trivial-rewrite[276]: warning: table socketmap:unix:/tmp/podop.socket:transport lookup error: Permission denied`
`smtp         | Dec 25 21:35:01 mail postfix/trivial-rewrite[276]: warning: socketmap:unix:/tmp/podop.socket:transport lookup error for "*"`
`smtp         | Dec 25 21:35:01 mail postfix/trivial-rewrite[276]: warning: connect to /tmp/podop.socket: Permission denied`
`smtp         | Dec 25 21:35:01 mail postfix/trivial-rewrite[276]: warning: table socketmap:unix:/tmp/podop.socket:domain lookup error: Permission denied`
`smtp         | Dec 25 21:35:01 mail postfix/trivial-rewrite[276]: warning: virtual_mailbox_domains: socketmap:unix:/tmp/podop.socket:domain: table lookup problem`
`smtp         | Dec 25 21:35:01 mail postfix/trivial-rewrite[276]: warning: virtual_mailbox_domains lookup failure`
`smtp         | Dec 25 21:35:19 mail postfix/trivial-rewrite[276]: warning: connect to /tmp/podop.socket: Permission denied`
`smtp         | Dec 25 21:35:19 mail postfix/trivial-rewrite[276]: warning: table socketmap:unix:/tmp/podop.socket:domain lookup error: Permission denied`
`smtp         | Dec 25 21:35:19 mail postfix/trivial-rewrite[276]: warning: virtual_mailbox_domains: socketmap:unix:/tmp/podop.socket:domain: table lookup problem`
`smtp         | Dec 25 21:35:19 mail postfix/trivial-rewrite[276]: warning: virtual_mailbox_domains lookup failure`
`smtp         | Dec 25 21:35:19 mail postfix/smtpd[198]: NOQUEUE: reject: RCPT from localhost[127.0.0.1]: 451 4.3.0 <tin@***obscured.com***>: Temporary lookup failure; from=<admin@***obscured.com***> to=<tin@***obscured.com***> proto=SMTP helo=<mail.***obscured.com***>`
`smtp         | Dec 25 21:35:24 mail postfix/smtpd[258]: connect from localhost[127.0.0.1]`
`smtp         | Dec 25 21:35:24 mail postfix/smtpd[258]: disconnect from localhost[127.0.0.1] quit=1 commands=1`


Discussion:
Looks like the Podop socket that is either not open or does not have the proper mode.

Could you enter the container and get us the output of `ls -la /tmp`, plus `ps faux`? 
Thanks for the comment @kaiyou.
Please see below.

`bash-5.0# ls -la /tmp/`
```
total 8
drwxrwxrwt    1 root     root          4096 Jan 11 10:02 .
drwxr-xr-x    1 root     root          4096 Jan 11 10:02 ..
srwxr-xr-x    1 messageb root             0 Jan 11 10:02 podop.socket
```

`bash-5.0# ps faux`
```
PID   USER     TIME  COMMAND
    1 root      0:01 {start.py} /usr/bin/python3 /start.py
   14 messageb  0:00 {start.py} /usr/bin/python3 /start.py
  111 root      0:00 {postfix-script} /bin/sh /usr/libexec/postfix/postfix-script start-fg
  178 root      0:00 /usr/libexec/postfix/master -s
  179 postfix   0:00 pickup -l -t unix -u
  180 postfix   0:00 qmgr -l -t unix -u
  181 postfix   0:00 postlogd -l -n postlog -t unix-dgram -u
  191 postfix   0:00 smtpd -n smtp -t inet -u -o stress= -s 2
  533 root      0:00 bash
  539 root      0:00 ps faux
```


I went ahead and did:
`chmod a+rwx /tmp/podop.socket`
And now I can both receive and send emails.

Could you please let me know the correct permissions?

Thanks,
Tin.
It should be owned by postfix, not this messageb user. Could you run `getent passwd` in the container?

Maybe the 100 uid changed recently. Which would lead to this catastrophe. 
Here you go. I included everything for completeness:
```
bash-5.0# getent passwd
root:x:0:0:root:/root:/bin/ash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
news:x:9:13:news:/usr/lib/news:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucppublic:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
man:x:13:15:man:/usr/man:/sbin/nologin
postmaster:x:14:12:postmaster:/var/spool/mail:/sbin/nologin
cron:x:16:16:cron:/var/spool/cron:/sbin/nologin
ftp:x:21:21::/var/lib/ftp:/sbin/nologin
sshd:x:22:22:sshd:/dev/null:/sbin/nologin
at:x:25:25:at:/var/spool/cron/atjobs:/sbin/nologin
squid:x:31:31:Squid:/var/cache/squid:/sbin/nologin
xfs:x:33:33:X Font Server:/etc/X11/fs:/sbin/nologin
games:x:35:35:games:/usr/games:/sbin/nologin
postgres:x:70:70::/var/lib/postgresql:/bin/sh
cyrus:x:85:12::/usr/cyrus:/sbin/nologin
vpopmail:x:89:89::/var/vpopmail:/sbin/nologin
ntp:x:123:123:NTP:/var/empty:/sbin/nologin
smmsp:x:209:209:smmsp:/var/spool/mqueue:/sbin/nologin
guest:x:405:100:guest:/dev/null:/sbin/nologin
nobody:x:65534:65534:nobody:/:/sbin/nologin
messagebus:x:100:101:messagebus:/dev/null:/sbin/nologin
postfix:x:101:102:postfix:/var/spool/postfix:/sbin/nologin
vmail:x:102:103:vmail:/var/mail/domains:/sbin/nologin

```
Oh dear this sounds bad. Which version of Mailu are you running? I will try to replicate this locally.

Possibly this is due to the base arm image which has a messagebus user that occupies uid 100. 
Yes, my arch is ARMv7.
I don't remember why I did, but I git clone and build the images locally.
See git history below.

`[bash]$ git log --pretty -p -1`

```
commit 5d20f2893aaa8b25a429ed2b895dee599c6f0de1 (HEAD -> master, origin/staging, origin/master, origin/HEAD)
Merge: 2b50333 0281205
Author: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Date:   Tue Dec 24 08:33:46 2019 +0000

    Merge #1291

    1291: Upgrade alpine to get new clamav. r=mergify[bot] a=youngnicks

    ## What type of PR?
    bug-fix

    ## What does this PR do?
    Upgrades alpine version to get new clamav. Also upgrades clamav config to latest variable names.

    ### Related issue(s)
    - Auto close an issue like: closes #1290


    Co-authored-by: Nick Young <nick@bublup.com>
```
That must be an issue with the different base image that installs a packages which reserves the postfix uid

We might switch image. In the meantime you could replace the uid 100 by uid 101 in the postfix start script. 
**cc** @abondis Maybe you have some insights (or know a better base image), since you did the ARM stuff?
unfortunately I don't have a better image. But I wonder, why is the script using a static UID? wouldn't there be a way to parametrize it? or use the username instead of the ID?
I do not think username is supported, but we could resolve it at startup. 
@kaiyou currently official `alpine` baseimage supports ARM architecture. I've built Mailu directly on ARM machine without build_arm.sh script and it works
@dmirubtsov , I did something naughty.
I pulled the latest git:
```
[bash]$ git log --pretty -p -1
commit 0bc10b7bc5b473b8b5f5c3286ba0b73f51fbc3ab (HEAD -> master, origin/staging, origin/master, origin/HEAD)
Merge: 9db7095 ae6f829
Author: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Date:   Wed Jan 29 18:59:50 2020 +0000

    Merge #1269

    1269: Add Fail2Ban section r=hoellen a=iohenkies

    ## What type of PR?
    documentation

    ## What does this PR do?
    This PR adds documentation to setup Fail2Ban to secure Mailu.

    ### Related issue(s)
    - closes #1263

    ## Prerequistes
    Before we can consider review and merge, please make sure the following list is done and checked.
    If an entry in not applicable, you can check it or remove it from the list.

    - [x] In case of feature or enhancement: documentation updated accordingly
    - [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.

    edit [@hoellen]: Give information about the PR and link issue.

    Co-authored-by: iohenkies <33115951+iohenkies@users.noreply.github.com>

```

and do a "docker-compose build" from the file "https://github.com/Mailu/Mailu/blob/master/tests/build.yml" file.
Running the container, and smtp server is met with an error:
`postsuper: fatal: scan_dir_push: open directory defer: Permission denied`


@kaiyou , on the same latest commit above. I ran "https://github.com/Mailu/Mailu/blob/master/tests/build_arm.sh".
Running the containers with the new images, I can send and receive emails without no errors.
No changes in socket permissions when I re-inspect them in the smtp container:

```
bash-5.0# getent passwd
.....
messagebus:x:100:101:messagebus:/dev/null:/sbin/nologin
postfix:x:101:102:postfix:/var/spool/postfix:/sbin/nologin
vmail:x:102:103:vmail:/var/mail/domains:/sbin/nologin

bash-5.0# ls /tmp/ -la
total 8
drwxrwxrwt    1 root     root          4096 Jan 30 00:32 .
drwxr-xr-x    1 root     root          4096 Jan 30 00:32 ..
srwxrwxrwx    1 messageb root             0 Jan 30 00:32 podop.socket
```

What do you think?

@dmirubtsov maybe we can get rid of some of the specifics of the arm build then, but I do not know how to build these on Travis.

@Tin-Bui I do not understand the issue in your last comment, could you elaborate ?
> currently official `alpine` baseimage supports ARM architecture. I've built Mailu directly on ARM machine without build_arm.sh script and it works

@dmirubtsov : the script was not meant to build for arm directly on arm, I don't know if it should work ... the purpose was to build the ARM images from a non-ARM platform (ie: powerful desktop vs doing it from a raspberry pi), that means using (or preparing) an image that has the `qemu-arm-static` binary (likeso https://ownyourbits.com/2018/06/27/running-and-building-arm-docker-containers-in-x86/)

> I do not know how to build these on Travis.

@kaiyou : is there no way to make travis run the script? It's been a while since I did it, but I think `docker run --rm --privileged multiarch/qemu-user-static:register` in the script makes it so the host doesn't need to have said binary locally.

@kaiyou https://github.com/Mailu/Mailu/issues/1294#issuecomment-580118006
I no longer have the original issue with permissions when sending and receiving emails.
No change to the original root cause (podop.socket permissions + users ID stay the same), but I no longer experience the symptom.
Probably a good thing?
Hi all,
I ran exactly in the same issue with the latest version of Mailu built with build_arm.sh.
uid 100 is messagebus... using a version of May 6...
```

 /hdd/dev/docker/Mailu.orig $ git log --pretty -p -1
commit d00ccc16d7d51be87daf761d7889ffa8cf059f23 (HEAD -> master, origin/staging, origin/master, origin/HEAD)
Merge: 535562e 455814d
Author: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Date:   Wed May 6 05:39:39 2020 +0000

    Merge #1487

    1487: Fix postfix queue permissions r=mergify[bot] a=ofthesun9

    ## What type of PR?
    bug-fix (by anticipation of a potential change in base image)

    ## What does this PR do?
    We need to check /queue permissions before starting postfix.
    In case of postfix/postdrop uid/gid change, postfix would fail to start

    ### Related issue(s)
    closes #1486

    ## Prerequistes
    - [x] In case of feature or enhancement: documentation updated accordingly


    Co-authored-by: ofthesun9 <olivier@ofthesun.net>
```

Did a chown postfix /tmp/podo.socket and all went good then...

I'll try building directly w/o the build_arm.sh script but I'm not sure why it would change anything: at the end, I will use the same alpine package, won't I?
It is weird indeed. I played a bit around the armbuild and never cross this issue, but I was building for arm64 not armhf...
I was unable to have antispam running on alpine:3.10 nor alpine:3.11 (SIGILL at startup), So I finally switched all the images to debian ;-)
If your platform can run arm64, and are willing to give it a try: my repo is ofthesun9 and the branch is test-debian-multiarch, or if you have a docker-compose.yml built from setup.mailu.io:

> docker-compose down 
> DOCKER_ORG=ofthesun9 MAILU_VERSION=test-debian-multiarch docker-compose up -d


Hi @ofthesun9, thanks
I just finished building the "standard" one (i.e. w/o the build_arm.sh script). I'll try it first and if it doesn't work, I'll give a try to yours and let you know.
As I am running a pi4, it can run arm64, I assume...
I had a look on the build_arm.sh script:
DISTRO is set to "balenalib/rpi-alpine:$ALPINE_VER"
I assume that balenalib/rpi-alpine is having an extra package already installed and taking the UID 100 before postfix get installed...
I built the standard version (with alpine 3.10.5) and everything seems to be working fine (including rspamd which is healthy)... 
Also, postfix uid is 100 and there is no messagebus user, so you're right @ofthesun9: I looked at the Docker file of rpi-alpine and it installs:
 apk add --no-cache bash ca-certificates curl dbus findutils tar udev gnupg
And dbus probably creates the messagebus user...
It means that it not safe at all to rely on an uid to build Mailu...

Maybe I'll push the mailu images for RPi on dockerhub later if my tests are ok...
@ofthesun9 
I am currently pushing my images to dockerhub if you want to give it a try :
DOCKER_ORG='leolivier' DOCKER_PREFIX='mailu_' MAILU_VERSION='rpi-armv7' docker-compose up -d

One thing I noticed compared to your Debian images is that mine (Alpine based) are much smaller (e.g. 30.13MB for mailu_postfix for me compared to 61.91MB - compressed - for your image tagged arm64-test-debian-multiarch)
Chiming in here as I hit the same issue (apparently because I built the ARM image on an ARM device).  Running this in the mailu directory (`/mailu` by default) did the trick for me:
`docker exec -it mailu_smtp_1 chown postfix /tmp/podop.socket`

Side note - I actually originally ran the build on a much more equipped laptop, but the docs were a bit lite - following https://mailu.io/master/rpi_build.html I ran `./deploy_to_pi.sh` and got an empty zip.  Frustration drove me to build directly on the pi...

Would love it if we had dockerhub images available!
If you are using an arm64 device, you can find images at #1421. If you are using an arm32 device, you have to generate your own build and follow tests/build_arm.sh 
Unfortunately, the use of balenalib/rpi-alpine as DISTRO is creating the podop.socket permission issue... the build_arm.sh script need an update ;-)
Could you please try to compile (on your amd64 laptop) using:
```
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
MAILU_VERSION=arm32v7_QEMU docker-compose -f tests/build.yml build --build-arg DISTRO=arm32v7/alpine:3.12  --build-arg ARCH=arm32v7/
```
Then you will need to push those images to your own dockerhub registry, and pull them back on your ARM device.
@fracture-point Not wanting to put myself in the spotlight :smile: but you can find on docker hub (leolivier) images for Mailu regularly updated (built on my Pi automatically every week for the core and monthly for the others)
@ofthesun9, yes the standard alpine image works like a charm.
Also I tried this weekend to change the Travis build to use the multiarch feature but unfortunately this works with arm64 but not arm32 :unamused:
Thank you both @ofthesun9 & @leolivier - your dockhub images are just what I needed (and also resolved another tricky issue I was trying to track down).
Hi There,

The `Mailu`-Project is currently in a bit of a bind! We are short on man-power, and we need to judge if it is possible for us to put in some work on this issue.

To help with that, we are currently trying to find out which issues are actively keeping users from using `Mailu`, which issues have someone who want to work on them — and which issues may be less important. These a less important ones could be discarded for the time being, until the project is in a more stable and regular state once again.

In order for us to better assess this, it would be helpful if you could put a **reaction on this post** (use the :smiley: icon to the top-right).
- 👍️ if you **need this** to be able to use Mailu. Ideally, you’d also be able to test this on your installation, and provide feedback …
- 🎉 if you find it a **nice bonus**, but no deal-breaker
- 🚀 if you want to **work on it yourself**!
We want to keep this **voting open for 2 weeks** from now, so please help out!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Hi There,

we see this issue had only little attention 😟. As much as it pains us: In order to get the current issues a bit more manageable for us, we decided to close it. 👋👋

We hope that no one feels offended by doing so. Should the issue get really pressing in the future, please feel free to re-open it.

Thank you for your patience and understanding, 🙇
- Your Mailu Team
This sounds like a real/current problem according to #1883 

Resolving patch:
diff --git a/core/postfix/start.py b/core/postfix/start.py
--- a/core/postfix/start.py
+++ b/core/postfix/start.py
@@ -8,12 +8,13 @@
 import sys
 
 from podop   import run_server
+from pwd import getpwnam
 from socrate import system, conf
 
 log.basicConfig(stream=sys.stderr, level=os.environ.get("LOG_LEVEL", "WARNING"))
 
 def start_podop():
-    os.setuid(100)
+    os.setuid(getpwnam('postfix').pw_uid)
     url = "http://" + os.environ["ADMIN_ADDRESS"] + "/internal/postfix/"
     # TODO: Remove verbosity setting from Podop?
     run_server(0, "postfix", "/tmp/podop.socket", [


Test patch: