Repository: moby/moby (language: go)

Issue:
Logs are not followed when using journald log-driver with systemd 255
### Description

When `journald` log-driver is used, running `docker logs --follow --tail` would not follow logs if started before any output is produced.

### Reproduce

1. Enable journald logging driver by putting `{ "log-driver": "journald" }` into `/etc/docker/daemon.json`.

2. Run the following script:

```sh
#!/bin/sh
trap 'docker kill logs-test' EXIT
docker kill logs-test

docker run --rm --name logs-test --detach alpine sh -c 'sleep 1; while sleep 0.1; do echo $((N++)); done'
# sleep 2
docker logs --follow --tail 1 logs-test
```

### Expected behavior

Expected result: it should print container output, i.e.,`0`,  `1`, `2`, `3`, and so on.

Actual result: it prints nothing. However, if you un-comment the `sleep 2` line, it will print the numbers as expected.

### docker version

```bash
Client:
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.21.3
 Git commit:        afdd53b4e3
 Built:             Sun Oct 29 15:42:02 2023
 OS/Arch:           linux/amd64
 Context:           default

Server:
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.21.3
  Git commit:       311b9ff0aa
  Built:            Sun Oct 29 15:42:02 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.7.11
  GitCommit:        64b8a811b07ba6288238eefc14d898ee0b5b99ba.m
 runc:
  Version:          1.1.10
  GitCommit:        
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
```


### docker info

```bash
Client:
 Version:    24.0.7
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: btrfs
  Supports d_type: true
  Using metacopy: true
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: journald
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 64b8a811b07ba6288238eefc14d898ee0b5b99ba.m
 runc version: 
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.6.7-arch1-1
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 1.926GiB
 Name: archlinux
 ID: bb0ac9f5-421b-45df-9718-ef4d1d752224
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
```


### Additional Info

_No response_


Discussion / hints:
Thanks for reporting! Do the daemon logs show anything useful in this case?

I wonder if it tries to follow logs from journald, but no journal is created yet if no logs were added yet (due to the initial `sleep 1`) 🤔 

https://github.com/moby/moby/blob/46f7ab808b9504d735d600e259ca0723f76fb164/daemon/logger/journald/read.go#L334-L342


However, there's also some other code that looks to have a timeout set https://github.com/moby/moby/blob/46f7ab808b9504d735d600e259ca0723f76fb164/daemon/logger/journald/read.go#L268-L272


I'm not deeply familiar with that driver, but perhaps @corhere has some ideas.

(daemon logs; if possible with `debug` enabled would be welcome though, in case those show interesting information)



Nothing interesting in the daemon logs as far as I can see.

<details><summary>Daemon log without sleep (no output)</summary>

```
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.283571880Z" level=debug msg="Calling POST /v1.43/containers/create?name=logs-test"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.283719659Z" level=debug msg="form data: {\"AttachStderr\":false,\"AttachStdin\":false,\"AttachStdout\":false,\"Cmd\":[\"sh\",\"-c\",\"sleep 1; while sleep 0.1; do echo $((N++)); done\"],\"Domainname\":\"\",\"Entrypoint\":null,\"Env\":null,\"HostConfig\":{\"AutoRemove\":true,\"Binds\":null,\"BlkioDeviceReadBps\":[],\"BlkioDeviceReadIOps\":[],\"BlkioDeviceWriteBps\":[],\"BlkioDeviceWriteIOps\":[],\"BlkioWeight\":0,\"BlkioWeightDevice\":[],\"CapAdd\":null,\"CapDrop\":null,\"Cgroup\":\"\",\"CgroupParent\":\"\",\"CgroupnsMode\":\"\",\"ConsoleSize\":[58,238],\"ContainerIDFile\":\"\",\"CpuCount\":0,\"CpuPercent\":0,\"CpuPeriod\":0,\"CpuQuota\":0,\"CpuRealtimePeriod\":0,\"CpuRealtimeRuntime\":0,\"CpuShares\":0,\"CpusetCpus\":\"\",\"CpusetMems\":\"\",\"DeviceCgroupRules\":null,\"DeviceRequests\":null,\"Devices\":[],\"Dns\":[],\"DnsOptions\":[],\"DnsSearch\":[],\"ExtraHosts\":null,\"GroupAdd\":null,\"IOMaximumBandwidth\":0,\"IOMaximumIOps\":0,\"IpcMode\":\"\",\"Isolation\":\"\",\"Links\":null,\"LogConfig\":{\"Config\":{},\"Type\":\"\"},\"MaskedPaths\":null,\"Memory\":0,\"MemoryReservation\":0,\"MemorySwap\":0,\"MemorySwappiness\":-1,\"NanoCpus\":0,\"NetworkMode\":\"default\",\"OomKillDisable\":false,\"OomScoreAdj\":0,\"PidMode\":\"\",\"PidsLimit\":0,\"PortBindings\":{},\"Privileged\":false,\"PublishAllPorts\":false,\"ReadonlyPaths\":null,\"ReadonlyRootfs\":false,\"RestartPolicy\":{\"MaximumRetryCount\":0,\"Name\":\"no\"},\"SecurityOpt\":null,\"ShmSize\":0,\"UTSMode\":\"\",\"Ulimits\":null,\"UsernsMode\":\"\",\"VolumeDriver\":\"\",\"VolumesFrom\":null},\"Hostname\":\"\",\"Image\":\"alpine\",\"Labels\":{},\"NetworkingConfig\":{\"EndpointsConfig\":{}},\"OnBuild\":null,\"OpenStdin\":false,\"StdinOnce\":false,\"Tty\":false,\"User\":\"\",\"Volumes\":{},\"WorkingDir\":\"\"}"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.387244480Z" level=debug msg="container mounted via layerStore: /var/lib/docker/overlay2/fa1e6b333149a7977d08ac48d4db5e7e74f0b76d8092b043fcdbaad01d8b170c/merged" container=534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.484934049Z" level=debug msg="Calling POST /v1.43/containers/534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829/wait?condition=removed"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.487936971Z" level=debug msg="Calling POST /v1.43/containers/534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829/start"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.490259399Z" level=debug msg="container mounted via layerStore: /var/lib/docker/overlay2/fa1e6b333149a7977d08ac48d4db5e7e74f0b76d8092b043fcdbaad01d8b170c/merged" container=534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.496073088Z" level=debug msg="Assigning addresses for endpoint logs-test's interface on network bridge"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.496129243Z" level=debug msg="RequestAddress(LocalDefault/172.17.0.0/16, <nil>, map[])"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.496157625Z" level=debug msg="Request address PoolID:172.17.0.0/16 Bits: 65536, Unselected: 65533, Sequence: (0xc0000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:3 Serial:false PrefAddress:invalid IP "
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.539178095Z" level=debug msg="Assigning addresses for endpoint logs-test's interface on network bridge"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.590221991Z" level=debug msg="Programming external connectivity on endpoint logs-test (53d1959a4254ef7493e93192801abe73cf44baee55c8cfc6fb69dc26bf16f6e3)"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.590997216Z" level=debug msg="DeleteConntrackEntries purged ipv4:0, ipv6:0"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.591082112Z" level=debug msg="DeleteConntrackEntriesByPort for udp ports purged ipv4:0, ipv6:0"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.605737235Z" level=debug msg="EnableService 534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829 START"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.605887555Z" level=debug msg="EnableService 534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829 DONE"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.613782947Z" level=debug msg="createSpec: cgroupsPath: system.slice:docker:534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.614135302Z" level=debug msg="bundle dir created" bundle=/var/run/docker/containerd/534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829 module=libcontainerd namespace=moby root=/var/lib/docker/overlay2/fa1e6b333149a7977d08ac48d4db5e7e74f0b76d8092b043fcdbaad01d8b170c/merged
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.811381040Z" level=debug msg="sandbox set key processing took 28.395654ms for container 534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.835274196Z" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/create
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.845751123Z" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/start
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.870524735Z" level=debug msg="Calling HEAD /_ping"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.870969739Z" level=debug msg="Calling GET /v1.43/containers/logs-test/json"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.871935140Z" level=debug msg="Calling GET /v1.43/containers/534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829/logs?follow=1&stderr=1&stdout=1&tail=1"
Dec 23 12:30:58 archlinux dockerd[1409]: time="2023-12-23T12:30:58.872846326Z" level=debug msg="begin logs" container=534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829 method="(*Daemon).ContainerLogs" module=daemon
Dec 23 12:30:59 archlinux 534ac7a0fb5f[1409]: 0
Dec 23 12:31:00 archlinux 534ac7a0fb5f[1409]: 1
Dec 23 12:31:00 archlinux 534ac7a0fb5f[1409]: 2
Dec 23 12:31:00 archlinux 534ac7a0fb5f[1409]: 3
Dec 23 12:31:00 archlinux 534ac7a0fb5f[1409]: 4
Dec 23 12:31:00 archlinux 534ac7a0fb5f[1409]: 5
Dec 23 12:31:00 archlinux 534ac7a0fb5f[1409]: 6
Dec 23 12:31:00 archlinux 534ac7a0fb5f[1409]: 7
Dec 23 12:31:00 archlinux 534ac7a0fb5f[1409]: 8
Dec 23 12:31:00 archlinux 534ac7a0fb5f[1409]: 9
Dec 23 12:31:00 archlinux dockerd[1409]: time="2023-12-23T12:31:00.976792361Z" level=debug msg="end logs (<nil>)" container=534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829 method="(*Daemon).ContainerLogs" module=daemon
Dec 23 12:31:00 archlinux 534ac7a0fb5f[1409]: 10
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.017695282Z" level=debug msg="Calling HEAD /_ping"
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.018156259Z" level=debug msg="Calling POST /v1.43/containers/logs-test/kill"
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.018294300Z" level=debug msg="Sending kill signal 9 to container 534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829"
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.040440065Z" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/exit
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.053960582Z" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/delete
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.053988074Z" level=info msg="ignoring event" container=534ac7a0fb5f22bcfb8cb5af5053f5d38087fa4437717d2065cc0f172f14d829 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.089649941Z" level=debug msg="Revoking external connectivity on endpoint logs-test (53d1959a4254ef7493e93192801abe73cf44baee55c8cfc6fb69dc26bf16f6e3)"
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.090410758Z" level=debug msg="DeleteConntrackEntries purged ipv4:0, ipv6:0"
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.090496489Z" level=debug msg="DeleteConntrackEntriesByPort for udp ports purged ipv4:0, ipv6:0"
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.185356783Z" level=debug msg="Releasing addresses for endpoint logs-test's interface on network bridge"
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.185393203Z" level=debug msg="ReleaseAddress(LocalDefault/172.17.0.0/16, 172.17.0.2)"
Dec 23 12:31:01 archlinux dockerd[1409]: time="2023-12-23T12:31:01.185413116Z" level=debug msg="Released address Address:172.17.0.2 Sequence:Bits: 65536, Unselected: 65533, Sequence: (0xc0000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:3"
```

</details>


<details><summary>Daemon log with sleep (produces output)</summary>

```
Dec 23 12:31:25 archlinux dockerd[1409]: time="2023-12-23T12:31:25.966660895Z" level=debug msg="Calling POST /v1.43/containers/create?name=logs-test"
Dec 23 12:31:25 archlinux dockerd[1409]: time="2023-12-23T12:31:25.966917624Z" level=debug msg="form data: {\"AttachStderr\":false,\"AttachStdin\":false,\"AttachStdout\":false,\"Cmd\":[\"sh\",\"-c\",\"sleep 1; while sleep 0.1; do echo $((N++)); done\"],\"Domainname\":\"\",\"Entrypoint\":null,\"Env\":null,\"HostConfig\":{\"AutoRemove\":true,\"Binds\":null,\"BlkioDeviceReadBps\":[],\"BlkioDeviceReadIOps\":[],\"BlkioDeviceWriteBps\":[],\"BlkioDeviceWriteIOps\":[],\"BlkioWeight\":0,\"BlkioWeightDevice\":[],\"CapAdd\":null,\"CapDrop\":null,\"Cgroup\":\"\",\"CgroupParent\":\"\",\"CgroupnsMode\":\"\",\"ConsoleSize\":[58,238],\"ContainerIDFile\":\"\",\"CpuCount\":0,\"CpuPercent\":0,\"CpuPeriod\":0,\"CpuQuota\":0,\"CpuRealtimePeriod\":0,\"CpuRealtimeRuntime\":0,\"CpuShares\":0,\"CpusetCpus\":\"\",\"CpusetMems\":\"\",\"DeviceCgroupRules\":null,\"DeviceRequests\":null,\"Devices\":[],\"Dns\":[],\"DnsOptions\":[],\"DnsSearch\":[],\"ExtraHosts\":null,\"GroupAdd\":null,\"IOMaximumBandwidth\":0,\"IOMaximumIOps\":0,\"IpcMode\":\"\",\"Isolation\":\"\",\"Links\":null,\"LogConfig\":{\"Config\":{},\"Type\":\"\"},\"MaskedPaths\":null,\"Memory\":0,\"MemoryReservation\":0,\"MemorySwap\":0,\"MemorySwappiness\":-1,\"NanoCpus\":0,\"NetworkMode\":\"default\",\"OomKillDisable\":false,\"OomScoreAdj\":0,\"PidMode\":\"\",\"PidsLimit\":0,\"PortBindings\":{},\"Privileged\":false,\"PublishAllPorts\":false,\"ReadonlyPaths\":null,\"ReadonlyRootfs\":false,\"RestartPolicy\":{\"MaximumRetryCount\":0,\"Name\":\"no\"},\"SecurityOpt\":null,\"ShmSize\":0,\"UTSMode\":\"\",\"Ulimits\":null,\"UsernsMode\":\"\",\"VolumeDriver\":\"\",\"VolumesFrom\":null},\"Hostname\":\"\",\"Image\":\"alpine\",\"Labels\":{},\"NetworkingConfig\":{\"EndpointsConfig\":{}},\"OnBuild\":null,\"OpenStdin\":false,\"StdinOnce\":false,\"Tty\":false,\"User\":\"\",\"Volumes\":{},\"WorkingDir\":\"\"}"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.058370322Z" level=debug msg="container mounted via layerStore: /var/lib/docker/overlay2/0a259d5c6d524330fa3bab0f81d4cb66be0be4e74d66a2d8bc84a4c57304b644/merged" container=6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.156884249Z" level=debug msg="Calling POST /v1.43/containers/6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839/wait?condition=removed"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.163262402Z" level=debug msg="Calling POST /v1.43/containers/6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839/start"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.166780319Z" level=debug msg="container mounted via layerStore: /var/lib/docker/overlay2/0a259d5c6d524330fa3bab0f81d4cb66be0be4e74d66a2d8bc84a4c57304b644/merged" container=6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.175131441Z" level=debug msg="Assigning addresses for endpoint logs-test's interface on network bridge"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.175184533Z" level=debug msg="RequestAddress(LocalDefault/172.17.0.0/16, <nil>, map[])"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.175260607Z" level=debug msg="Request address PoolID:172.17.0.0/16 Bits: 65536, Unselected: 65533, Sequence: (0xc0000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:3 Serial:false PrefAddress:invalid IP "
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.227197617Z" level=debug msg="Assigning addresses for endpoint logs-test's interface on network bridge"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.325312055Z" level=debug msg="Programming external connectivity on endpoint logs-test (b7859c8104fd1eca1432f2c61f1f207477c1bf2fafa84e4968261ea57e5db947)"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.329249530Z" level=debug msg="DeleteConntrackEntries purged ipv4:0, ipv6:0"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.329373016Z" level=debug msg="DeleteConntrackEntriesByPort for udp ports purged ipv4:0, ipv6:0"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.379085777Z" level=debug msg="EnableService 6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839 START"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.379219174Z" level=debug msg="EnableService 6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839 DONE"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.403106310Z" level=debug msg="createSpec: cgroupsPath: system.slice:docker:6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.407671350Z" level=debug msg="bundle dir created" bundle=/var/run/docker/containerd/6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839 module=libcontainerd namespace=moby root=/var/lib/docker/overlay2/0a259d5c6d524330fa3bab0f81d4cb66be0be4e74d66a2d8bc84a4c57304b644/merged
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.619996401Z" level=debug msg="sandbox set key processing took 25.853865ms for container 6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839"
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.642902643Z" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/create
Dec 23 12:31:26 archlinux dockerd[1409]: time="2023-12-23T12:31:26.654045342Z" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/start
Dec 23 12:31:27 archlinux 6e1b77fb42e9[1409]: 0
Dec 23 12:31:27 archlinux 6e1b77fb42e9[1409]: 1
Dec 23 12:31:27 archlinux 6e1b77fb42e9[1409]: 2
Dec 23 12:31:28 archlinux 6e1b77fb42e9[1409]: 3
Dec 23 12:31:28 archlinux 6e1b77fb42e9[1409]: 4
Dec 23 12:31:28 archlinux 6e1b77fb42e9[1409]: 5
Dec 23 12:31:28 archlinux 6e1b77fb42e9[1409]: 6
Dec 23 12:31:28 archlinux 6e1b77fb42e9[1409]: 7
Dec 23 12:31:28 archlinux 6e1b77fb42e9[1409]: 8
Dec 23 12:31:28 archlinux 6e1b77fb42e9[1409]: 9
Dec 23 12:31:28 archlinux dockerd[1409]: time="2023-12-23T12:31:28.710567834Z" level=debug msg="Calling HEAD /_ping"
Dec 23 12:31:28 archlinux dockerd[1409]: time="2023-12-23T12:31:28.711280124Z" level=debug msg="Calling GET /v1.43/containers/logs-test/json"
Dec 23 12:31:28 archlinux dockerd[1409]: time="2023-12-23T12:31:28.713491872Z" level=debug msg="Calling GET /v1.43/containers/6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839/logs?follow=1&stderr=1&stdout=1&tail=1"
Dec 23 12:31:28 archlinux dockerd[1409]: time="2023-12-23T12:31:28.714827012Z" level=debug msg="begin logs" container=6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839 method="(*Daemon).ContainerLogs" module=daemon
Dec 23 12:31:28 archlinux 6e1b77fb42e9[1409]: 10
Dec 23 12:31:28 archlinux 6e1b77fb42e9[1409]: 11
Dec 23 12:31:29 archlinux 6e1b77fb42e9[1409]: 12
Dec 23 12:31:29 archlinux 6e1b77fb42e9[1409]: 13
Dec 23 12:31:29 archlinux 6e1b77fb42e9[1409]: 14
Dec 23 12:31:29 archlinux 6e1b77fb42e9[1409]: 15
Dec 23 12:31:29 archlinux 6e1b77fb42e9[1409]: 16
Dec 23 12:31:29 archlinux 6e1b77fb42e9[1409]: 17
Dec 23 12:31:29 archlinux 6e1b77fb42e9[1409]: 18
Dec 23 12:31:29 archlinux 6e1b77fb42e9[1409]: 19
Dec 23 12:31:29 archlinux dockerd[1409]: time="2023-12-23T12:31:29.828742739Z" level=debug msg="end logs (<nil>)" container=6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839 method="(*Daemon).ContainerLogs" module=daemon
Dec 23 12:31:29 archlinux 6e1b77fb42e9[1409]: 20
Dec 23 12:31:29 archlinux dockerd[1409]: time="2023-12-23T12:31:29.870388282Z" level=debug msg="Calling HEAD /_ping"
Dec 23 12:31:29 archlinux dockerd[1409]: time="2023-12-23T12:31:29.871051919Z" level=debug msg="Calling POST /v1.43/containers/logs-test/kill"
Dec 23 12:31:29 archlinux dockerd[1409]: time="2023-12-23T12:31:29.871110691Z" level=debug msg="Sending kill signal 9 to container 6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839"
Dec 23 12:31:29 archlinux dockerd[1409]: time="2023-12-23T12:31:29.901152398Z" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/exit
Dec 23 12:31:29 archlinux dockerd[1409]: time="2023-12-23T12:31:29.913318543Z" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/delete
Dec 23 12:31:29 archlinux dockerd[1409]: time="2023-12-23T12:31:29.913350782Z" level=info msg="ignoring event" container=6e1b77fb42e915f36d65e39885765b9a72270eb96a12a994478cda20bdbaa839 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Dec 23 12:31:29 archlinux dockerd[1409]: time="2023-12-23T12:31:29.943009470Z" level=debug msg="Revoking external connectivity on endpoint logs-test (b7859c8104fd1eca1432f2c61f1f207477c1bf2fafa84e4968261ea57e5db947)"
Dec 23 12:31:29 archlinux dockerd[1409]: time="2023-12-23T12:31:29.943773642Z" level=debug msg="DeleteConntrackEntries purged ipv4:0, ipv6:0"
Dec 23 12:31:29 archlinux dockerd[1409]: time="2023-12-23T12:31:29.943860589Z" level=debug msg="DeleteConntrackEntriesByPort for udp ports purged ipv4:0, ipv6:0"
Dec 23 12:31:30 archlinux dockerd[1409]: time="2023-12-23T12:31:30.046885931Z" level=debug msg="Releasing addresses for endpoint logs-test's interface on network bridge"
Dec 23 12:31:30 archlinux dockerd[1409]: time="2023-12-23T12:31:30.047061747Z" level=debug msg="ReleaseAddress(LocalDefault/172.17.0.0/16, 172.17.0.2)"
Dec 23 12:31:30 archlinux dockerd[1409]: time="2023-12-23T12:31:30.047088706Z" level=debug msg="Released address Address:172.17.0.2 Sequence:Bits: 65536, Unselected: 65533, Sequence: (0xc0000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:3"
```

</details>

---

Related/reduced cases:

### Docker API

Same behavior (with and without sleep).

```sh
docker run --rm --name logs-test --detach alpine sh -c 'sleep 1; while sleep 0.1; do echo $((N++)); done'
# sleep 2
curl --output - --silent --unix-socket /run/docker.sock "http://v1.43/containers/logs-test/logs?follow=1&stdout=1&tail=1"
```

### Journalctl

Works almost as expected, however it starts from `2` rather than from `0` for some reason.

```sh
D=$(docker run --rm --name logs-test --detach alpine sh -c 'sleep 1; while sleep 0.1; do echo $((N++)); done')
# sleep 2
journalctl -f CONTAINER_ID_FULL=$D
```
What version of systemd are you using? And what's the output of `ldd $(which dockerd)`?
Everything is from the repos on a newly created Arch Linux VM.

```console
$ systemctl --version
systemd 255 (255.1-1-arch)
+PAM +AUDIT -SELINUX -APPARMOR -IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP -SYSVINIT default-hierarchy=unified
```

```console
$ ldd $(which dockerd)
	linux-vdso.so.1 (0x00007ffc532a0000)
	libsystemd.so.0 => /usr/lib/libsystemd.so.0 (0x00007f2255548000)
	libdevmapper.so.1.02 => /usr/lib/libdevmapper.so.1.02 (0x00007f22509a4000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f22507c2000)
	libcap.so.2 => /usr/lib/libcap.so.2 (0x00007f225553c000)
	libgcrypt.so.20 => /usr/lib/libgcrypt.so.20 (0x00007f2250679000)
	liblz4.so.1 => /usr/lib/liblz4.so.1 (0x00007f2250657000)
	liblzma.so.5 => /usr/lib/liblzma.so.5 (0x00007f2250624000)
	libzstd.so.1 => /usr/lib/libzstd.so.1 (0x00007f2250551000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f225052c000)
	/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f2255644000)
	libudev.so.1 => /usr/lib/libudev.so.1 (0x00007f22504f5000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007f2250408000)
	libgpg-error.so.0 => /usr/lib/libgpg-error.so.0 (0x00007f22503e2000)
```

Fix patch:
diff --git a/daemon/logger/journald/internal/fake/sender.go b/daemon/logger/journald/internal/fake/sender.go
index 0d4bf8e90ea68..050f1b71ad407 100644
--- a/daemon/logger/journald/internal/fake/sender.go
+++ b/daemon/logger/journald/internal/fake/sender.go
@@ -22,6 +22,7 @@ import (
 
 	"code.cloudfoundry.org/clock"
 	"github.com/coreos/go-systemd/v22/journal"
+	"github.com/google/uuid"
 	"gotest.tools/v3/assert"
 
 	"github.com/docker/docker/daemon/logger/journald/internal/export"
@@ -67,6 +68,14 @@ type Sender struct {
 	// timestamp in zero time after the SYSLOG_TIMESTAMP value was set,
 	// which is higly unrealistic in practice.
 	AssignEventTimestampFromSyslogTimestamp bool
+	// Boot ID for journal entries. Required by systemd-journal-remote as of
+	// https://github.com/systemd/systemd/commit/1eede158519e4e5ed22738c90cb57a91dbecb7f2
+	// (systemd 255).
+	BootID uuid.UUID
+
+	// When set, Send will act as a test helper and redirect
+	// systemd-journal-remote command output to the test log.
+	TB testing.TB
 }
 
 // New constructs a new Sender which will write journal entries to outpath. The
@@ -82,6 +91,7 @@ func New(outpath string) (*Sender, error) {
 		CmdName:    p,
 		OutputPath: outpath,
 		Clock:      clock.NewClock(),
+		BootID:     uuid.New(), // UUIDv4, like systemd itself generates for sd_id128 values.
 	}
 	return sender, nil
 }
@@ -95,6 +105,7 @@ func NewT(t *testing.T, outpath string) *Sender {
 		t.Skip(err)
 	}
 	assert.NilError(t, err)
+	s.TB = t
 	return s
 }
 
@@ -103,6 +114,9 @@ var validVarName = regexp.MustCompile("^[A-Z0-9][A-Z0-9_]*$")
 // Send is a drop-in replacement for
 // github.com/coreos/go-systemd/v22/journal.Send.
 func (s *Sender) Send(message string, priority journal.Priority, vars map[string]string) error {
+	if s.TB != nil {
+		s.TB.Helper()
+	}
 	var buf bytes.Buffer
 	// https://systemd.io/JOURNAL_EXPORT_FORMATS/ says "if you are
 	// generating this format you shouldn’t care about these special
@@ -121,6 +135,9 @@ func (s *Sender) Send(message string, priority journal.Priority, vars map[string
 	if err := export.WriteField(&buf, "__REALTIME_TIMESTAMP", strconv.FormatInt(ts.UnixMicro(), 10)); err != nil {
 		return fmt.Errorf("fake: error writing entry to systemd-journal-remote: %w", err)
 	}
+	if err := export.WriteField(&buf, "_BOOT_ID", fmt.Sprintf("%x", [16]byte(s.BootID))); err != nil {
+		return fmt.Errorf("fake: error writing entry to systemd-journal-remote: %w", err)
+	}
 	if err := export.WriteField(&buf, "MESSAGE", message); err != nil {
 		return fmt.Errorf("fake: error writing entry to systemd-journal-remote: %w", err)
 	}
@@ -143,6 +160,16 @@ func (s *Sender) Send(message string, priority journal.Priority, vars map[string
 	// has been flushed to disk when Send returns.
 	cmd := exec.Command(s.CmdName, "--output", s.OutputPath, "-")
 	cmd.Stdin = &buf
+
+	if s.TB != nil {
+		out, err := cmd.CombinedOutput()
+		s.TB.Logf("[systemd-journal-remote] %s", out)
+		var exitErr *exec.ExitError
+		if errors.As(err, &exitErr) {
+			s.TB.Logf("systemd-journal-remote exit status: %d", exitErr.ExitCode())
+		}
+		return err
+	}
 	cmd.Stdout = os.Stdout
 	cmd.Stderr = os.Stderr
 	return cmd.Run()
diff --git a/daemon/logger/journald/read.go b/daemon/logger/journald/read.go
index 6e057b5179250..96e0dea68f195 100644
--- a/daemon/logger/journald/read.go
+++ b/daemon/logger/journald/read.go
@@ -4,7 +4,6 @@ package journald // import "github.com/docker/docker/daemon/logger/journald"
 
 import (
 	"context"
-	"errors"
 	"runtime"
 	"strconv"
 	"sync/atomic"
@@ -17,7 +16,10 @@ import (
 	"github.com/docker/docker/daemon/logger/journald/internal/sdjournal"
 )
 
-const closedDrainTimeout = 5 * time.Second
+const (
+	closedDrainTimeout = 5 * time.Second
+	waitInterval       = 250 * time.Millisecond
+)
 
 // Fields which we know are not user-provided attribute fields.
 var wellKnownFields = map[string]bool{
@@ -46,13 +48,13 @@ var wellKnownFields = map[string]bool{
 }
 
 type reader struct {
-	s           *journald
-	j           *sdjournal.Journal
-	logWatcher  *logger.LogWatcher
-	config      logger.ReadConfig
-	maxOrdinal  uint64
-	initialized bool
-	ready       chan struct{}
+	s             *journald
+	j             *sdjournal.Journal
+	logWatcher    *logger.LogWatcher
+	config        logger.ReadConfig
+	maxOrdinal    uint64
+	ready         chan struct{}
+	drainDeadline time.Time
 }
 
 func getMessage(d map[string]string) (line []byte, ok bool) {
@@ -99,101 +101,168 @@ func getAttrs(d map[string]string) []backend.LogAttr {
 	return attrs
 }
 
-// errDrainDone is the error returned by drainJournal to signal that there are
-// no more log entries to send to the log watcher.
-var errDrainDone = errors.New("journald drain done")
+// The SeekXYZ() methods all move the journal read pointer to a "conceptual"
+// position which does not correspond to any journal entry. A subsequent call to
+// Next(), Previous() or similar is necessary to resolve the read pointer to a
+// discrete entry.
+// https://github.com/systemd/systemd/pull/5930#issuecomment-300878104
+// But that's not all! If there is no discrete entry to resolve the position to,
+// the call to Next() or Previous() will just leave the read pointer in a
+// conceptual position, or do something even more bizarre.
+// https://github.com/systemd/systemd/issues/9934
+
+// initialSeekHead positions the journal read pointer at the earliest journal
+// entry with a timestamp of at least r.config.Since. It returns true if there
+// is an entry to read at the read pointer.
+func (r *reader) initialSeekHead() (bool, error) {
+	var err error
+	if r.config.Since.IsZero() {
+		err = r.j.SeekHead()
+	} else {
+		err = r.j.SeekRealtime(r.config.Since)
+	}
+	if err != nil {
+		return false, err
+	}
+	return r.j.Next()
+}
 
-// drainJournal reads and sends log messages from the journal.
-//
-// drainJournal returns errDrainDone when a terminal stopping condition has been
-// reached: the watch consumer is gone, a log entry is read which has a
-// timestamp after until (if until is nonzero), or the log driver is closed and
-// the last message logged has been sent from the journal. If the end of the
-// journal is reached without encountering a terminal stopping condition, a nil
-// error is returned.
-func (r *reader) drainJournal() error {
-	if !r.initialized {
-		defer func() {
-			r.signalReady()
-			r.initialized = true
-		}()
+// initialSeekTail positions the journal read pointer at a journal entry
+// relative to the tail of the journal at the time of the call based on the
+// specification in r.config. It returns true if there is an entry to read at
+// the read pointer. Otherwise the read pointer is set to a conceptual position
+// which will be resolved to the desired entry (once written) by advancing
+// forward with r.j.Next() or similar.
+func (r *reader) initialSeekTail() (bool, error) {
+	var err error
+	if r.config.Until.IsZero() {
+		err = r.j.SeekTail()
+	} else {
+		err = r.j.SeekRealtime(r.config.Until)
+	}
+	if err != nil {
+		return false, err
+	}
 
-		var (
-			err          error
-			seekedToTail bool
-		)
-		if r.config.Tail >= 0 {
-			if r.config.Until.IsZero() {
-				err = r.j.SeekTail()
-				seekedToTail = true
-			} else {
-				err = r.j.SeekRealtime(r.config.Until)
-			}
-		} else {
-			if r.config.Since.IsZero() {
-				err = r.j.SeekHead()
-			} else {
-				err = r.j.SeekRealtime(r.config.Since)
+	var ok bool
+	if r.config.Tail == 0 {
+		ok, err = r.j.Previous()
+	} else {
+		var n int
+		n, err = r.j.PreviousSkip(uint(r.config.Tail))
+		ok = n > 0
+	}
+	if err != nil {
+		return ok, err
+	}
+	if !ok {
+		// The (filtered) journal has no entries. The tail is the head: all new
+		// entries which get written into the journal from this point forward
+		// should be read from the journal. However the read pointer is
+		// positioned at a conceptual position which is not condusive to reading
+		// those entries. The tail of the journal is resolved to the last entry
+		// in the journal _at the time of the first successful Previous() call_,
+		// which means that an arbitrary number of journal entries added in the
+		// interim may be skipped: race condition. While the realtime conceptual
+		// position is not so racy, it is also unhelpful: it is the timestamp
+		// past where reading should stop, so all logs that should be followed
+		// would be skipped over.
+		// Reset the read pointer position to avoid these problems.
+		return r.initialSeekHead()
+	} else if r.config.Tail == 0 {
+		// The journal read pointer is positioned at the discrete position of
+		// the journal entry _before_ the entry to send.
+		return r.j.Next()
+	}
+
+	// Check if the PreviousSkip went too far back.
+	timestamp, err := r.j.Realtime()
+	if err != nil {
+		return false, err
+	}
+	if timestamp.Before(r.config.Since) {
+		if err := r.j.SeekRealtime(r.config.Since); err != nil {
+			return false, err
+		}
+		return r.j.Next()
+	}
+	return true, nil
+}
+
+// wait blocks until the journal has new data to read, the reader's drain
+// deadline is exceeded, or the log reading consumer is gone.
+func (r *reader) wait() (bool, error) {
+	for {
+		dur := waitInterval
+		if !r.drainDeadline.IsZero() {
+			dur = time.Until(r.drainDeadline)
+			if dur < 0 {
+				// Container is gone but we haven't found the end of the
+				// logs before the deadline. Maybe it was dropped by
+				// journald, e.g. due to rate-limiting.
+				return false, nil
+			} else if dur > waitInterval {
+				dur = waitInterval
 			}
 		}
+		status, err := r.j.Wait(dur)
 		if err != nil {
-			return err
+			return false, err
+		} else if status != sdjournal.StatusNOP {
+			return true, nil
 		}
-
-		// SeekTail() followed by Next() behaves incorrectly, so we need
-		// to work around the bug by ensuring the first discrete
-		// movement of the read pointer is Previous() or PreviousSkip().
-		// PreviousSkip() is called inside the loop when config.Tail > 0
-		// so the only special case requiring special handling is
-		// config.Tail == 0.
-		// https://github.com/systemd/systemd/issues/9934
-		if seekedToTail && r.config.Tail == 0 {
-			// Resolve the read pointer to the last entry in the
-			// journal so that the call to Next() inside the loop
-			// advances past it.
-			if ok, err := r.j.Previous(); err != nil || !ok {
-				return err
+		select {
+		case <-r.logWatcher.WatchConsumerGone():
+			return false, nil
+		case <-r.s.closed:
+			// Container is gone; don't wait indefinitely for journal entries that will never arrive.
+			if r.maxOrdinal >= atomic.LoadUint64(&r.s.ordinal) {
+				return false, nil
+			}
+			if r.drainDeadline.IsZero() {
+				r.drainDeadline = time.Now().Add(closedDrainTimeout)
 			}
+		default:
 		}
 	}
+}
 
-	for i := 0; ; i++ {
-		if !r.initialized && i == 0 && r.config.Tail > 0 {
-			if n, err := r.j.PreviousSkip(uint(r.config.Tail)); err != nil || n == 0 {
-				return err
-			}
-		} else if ok, err := r.j.Next(); err != nil || !ok {
-			return err
+// nextWait blocks until there is a new journal entry to read, and advances the
+// journal read pointer to it.
+func (r *reader) nextWait() (bool, error) {
+	for {
+		if ok, err := r.j.Next(); err != nil || ok {
+			return ok, err
 		}
-
-		if !r.initialized && i == 0 {
-			// The cursor is in a position which will be unaffected
-			// by subsequent logging.
-			r.signalReady()
+		if ok, err := r.wait(); err != nil || !ok {
+			return false, err
 		}
+	}
+}
 
+// drainJournal reads and sends log messages from the journal, starting from the
+// current read pointer, until the end of the journal or a terminal stopping
+// condition is reached.
+//
+// It returns false when a terminal stopping condition has been reached:
+//   - the watch consumer is gone, or
+//   - (if until is nonzero) a log entry is read which has a timestamp after
+//     until
+func (r *reader) drainJournal() (bool, error) {
+	for i := 0; ; i++ {
 		// Read the entry's timestamp.
 		timestamp, err := r.j.Realtime()
 		if err != nil {
-			return err
-		}
-		// Check if the PreviousSkip went too far back. Check only the
-		// initial position as we are comparing wall-clock timestamps,
-		// which may not be monotonic. We don't want to skip over
-		// messages sent later in time just because the clock moved
-		// backwards.
-		if !r.initialized && i == 0 && r.config.Tail > 0 && timestamp.Before(r.config.Since) {
-			r.j.SeekRealtime(r.config.Since)
-			continue
+			return true, err
 		}
 		if !r.config.Until.IsZero() && r.config.Until.Before(timestamp) {
-			return errDrainDone
+			return false, nil
 		}
 
 		// Read and send the logged message, if there is one to read.
 		data, err := r.j.Data()
 		if err != nil {
-			return err
+			return true, err
 		}
 
 		if data[fieldLogEpoch] == r.s.epoch {
@@ -228,7 +297,7 @@ func (r *reader) drainJournal() error {
 			*/
 			select {
 			case <-r.logWatcher.WatchConsumerGone():
-				return errDrainDone
+				return false, nil
 			case r.logWatcher.Msg <- msg:
 			}
 		}
@@ -243,41 +312,28 @@ func (r *reader) drainJournal() error {
 					Warn("journald: error processing journal")
 			}
 		}
+
+		if ok, err := r.j.Next(); err != nil || !ok {
+			return true, err
+		}
 	}
 }
 
 func (r *reader) readJournal() error {
 	caughtUp := atomic.LoadUint64(&r.s.ordinal)
-	if err := r.drainJournal(); err != nil {
-		if err != errDrainDone {
-			return err
-		}
-		return nil
+	if more, err := r.drainJournal(); err != nil || !more {
+		return err
 	}
 
-	var drainTimeout <-chan time.Time
 	if !r.config.Follow {
 		if r.s.readSyncTimeout == 0 {
 			return nil
 		}
-		tmr := time.NewTimer(r.s.readSyncTimeout)
-		defer tmr.Stop()
-		drainTimeout = tmr.C
+		r.drainDeadline = time.Now().Add(r.s.readSyncTimeout)
 	}
 
 	for {
-		status, err := r.j.Wait(250 * time.Millisecond)
-		if err != nil {
-			return err
-		}
 		select {
-		case <-r.logWatcher.WatchConsumerGone():
-			return nil // won't be able to write anything anymore
-		case <-drainTimeout:
-			// Container is gone but we haven't found the end of the
-			// logs within the timeout. Maybe it was dropped by
-			// journald, e.g. due to rate-limiting.
-			return nil
 		case <-r.s.closed:
 			// container is gone, drain journal
 			lastSeq := atomic.LoadUint64(&r.s.ordinal)
@@ -285,24 +341,14 @@ func (r *reader) readJournal() error {
 				// All caught up with the logger!
 				return nil
 			}
-			if drainTimeout == nil {
-				tmr := time.NewTimer(closedDrainTimeout)
-				defer tmr.Stop()
-				drainTimeout = tmr.C
-			}
 		default:
-			// container is still alive
-			if status == sdjournal.StatusNOP {
-				// no new data -- keep waiting
-				continue
-			}
 		}
-		err = r.drainJournal()
-		if err != nil {
-			if err != errDrainDone {
-				return err
-			}
-			return nil
+
+		if more, err := r.nextWait(); err != nil || !more {
+			return err
+		}
+		if more, err := r.drainJournal(); err != nil || !more {
+			return err
 		}
 		if !r.config.Follow && r.s.readSyncTimeout > 0 && r.maxOrdinal >= caughtUp {
 			return nil
@@ -361,6 +407,33 @@ func (r *reader) readLogs() {
 		return
 	}
 
+	var ok bool
+	if r.config.Tail >= 0 {
+		ok, err = r.initialSeekTail()
+	} else {
+		ok, err = r.initialSeekHead()
+	}
+	if err != nil {
+		r.logWatcher.Err <- err
+		return
+	}
+	r.signalReady()
+	if !ok {
+		if !r.config.Follow {
+			return
+		}
+		// Either the read pointer is positioned at a discrete journal entry, in
+		// which case the position will be unaffected by subsequent logging, or
+		// the read pointer is in the conceptual position corresponding to the
+		// first journal entry to send once it is logged in the future.
+		if more, err := r.nextWait(); err != nil || !more {
+			if err != nil {
+				r.logWatcher.Err <- err
+			}
+			return
+		}
+	}
+
 	if err := r.readJournal(); err != nil {
 		r.logWatcher.Err <- err
 		return


Test patch:
diff --git a/daemon/logger/journald/read_test.go b/daemon/logger/journald/read_test.go
index 86fc6b280d83c..771057671cb80 100644
--- a/daemon/logger/journald/read_test.go
+++ b/daemon/logger/journald/read_test.go
@@ -3,6 +3,7 @@
 package journald // import "github.com/docker/docker/daemon/logger/journald"
 
 import (
+	"sync"
 	"testing"
 	"time"
 
@@ -46,32 +47,37 @@ func TestLogRead(t *testing.T) {
 			assert.NilError(t, rotatedJournal.Send("a log message from a totally different process in the active journal", journal.PriInfo, nil))
 
 			return func(t *testing.T) logger.Logger {
+				l, err := new(info)
+				assert.NilError(t, err)
+				l.journalReadDir = journalDir
+				sl := &syncLogger{journald: l, waiters: map[uint64]chan<- struct{}{}}
+
 				s := make(chan sendit, 100)
 				t.Cleanup(func() { close(s) })
 				go func() {
 					for m := range s {
 						<-m.after
 						activeJournal.Send(m.message, m.priority, m.vars)
-						if m.sent != nil {
-							close(m.sent)
+						sl.mu.Lock()
+						sl.sent++
+						if notify, ok := sl.waiters[sl.sent]; ok {
+							delete(sl.waiters, sl.sent)
+							close(notify)
 						}
+						sl.mu.Unlock()
 					}
 				}()
-				l, err := new(info)
-				assert.NilError(t, err)
-				l.journalReadDir = journalDir
 
-				sl := &syncLogger{journald: l}
 				l.sendToJournal = func(message string, priority journal.Priority, vars map[string]string) error {
-					sent := make(chan struct{})
+					sl.mu.Lock()
+					sl.queued++
+					sl.mu.Unlock()
 					s <- sendit{
 						message:  message,
 						priority: priority,
 						vars:     vars,
 						after:    time.After(150 * time.Millisecond),
-						sent:     sent,
 					}
-					sl.waitOn = sent
 					return nil
 				}
 				l.readSyncTimeout = 3 * time.Second
@@ -88,17 +94,31 @@ type sendit struct {
 	priority journal.Priority
 	vars     map[string]string
 	after    <-chan time.Time
-	sent     chan<- struct{}
 }
 
 type syncLogger struct {
 	*journald
-	waitOn <-chan struct{}
+
+	mu           sync.Mutex
+	queued, sent uint64
+	waiters      map[uint64]chan<- struct{}
 }
 
 func (l *syncLogger) Sync() error {
-	if l.waitOn != nil {
-		<-l.waitOn
+	l.mu.Lock()
+	waitFor := l.queued
+	if l.sent >= l.queued {
+		l.mu.Unlock()
+		return nil
 	}
+	notify := make(chan struct{})
+	l.waiters[waitFor] = notify
+	l.mu.Unlock()
+	<-notify
 	return nil
 }
+
+func (l *syncLogger) Close() error {
+	_ = l.Sync()
+	return l.journald.Close()
+}
diff --git a/daemon/logger/loggertest/logreader.go b/daemon/logger/loggertest/logreader.go
index d00b6a2484693..0573a6add4af7 100644
--- a/daemon/logger/loggertest/logreader.go
+++ b/daemon/logger/loggertest/logreader.go
@@ -1,6 +1,7 @@
 package loggertest // import "github.com/docker/docker/daemon/logger/loggertest"
 
 import (
+	"fmt"
 	"runtime"
 	"strings"
 	"sync"
@@ -10,6 +11,7 @@ import (
 	"github.com/google/go-cmp/cmp"
 	"github.com/google/go-cmp/cmp/cmpopts"
 	"gotest.tools/v3/assert"
+	is "gotest.tools/v3/assert/cmp"
 	"gotest.tools/v3/assert/opt"
 
 	"github.com/docker/docker/api/types/backend"
@@ -194,28 +196,31 @@ func (tr Reader) testTailEmptyLogs(t *testing.T, live bool) {
 func (tr Reader) TestFollow(t *testing.T) {
 	// Reader sends all logs and closes after logger is closed
 	// - Starting from empty log (like run)
-	t.Run("FromEmptyLog", func(t *testing.T) {
-		t.Parallel()
-		l := tr.Factory(t, logger.Info{
-			ContainerID:   "followstart0",
-			ContainerName: "logloglog",
-		})(t)
-		lw := l.(logger.LogReader).ReadLogs(logger.ReadConfig{Tail: -1, Follow: true})
-		defer lw.ConsumerGone()
-
-		doneReading := make(chan struct{})
-		var logs []*logger.Message
-		go func() {
-			defer close(doneReading)
-			logs = readAll(t, lw)
-		}()
+	for i, tail := range []int{-1, 0, 1, 42} {
+		i, tail := i, tail
+		t.Run(fmt.Sprintf("FromEmptyLog/Tail=%d", tail), func(t *testing.T) {
+			t.Parallel()
+			l := tr.Factory(t, logger.Info{
+				ContainerID:   fmt.Sprintf("followstart%d", i),
+				ContainerName: fmt.Sprintf("logloglog%d", i),
+			})(t)
+			lw := l.(logger.LogReader).ReadLogs(logger.ReadConfig{Tail: tail, Follow: true})
+			defer lw.ConsumerGone()
 
-		mm := makeTestMessages()
-		expected := logMessages(t, l, mm)
-		assert.NilError(t, l.Close())
-		<-doneReading
-		assert.DeepEqual(t, logs, expected, compareLog)
-	})
+			doneReading := make(chan struct{})
+			var logs []*logger.Message
+			go func() {
+				defer close(doneReading)
+				logs = readAll(t, lw)
+			}()
+
+			mm := makeTestMessages()
+			expected := logMessages(t, l, mm)
+			assert.NilError(t, l.Close())
+			<-doneReading
+			assert.DeepEqual(t, logs, expected, compareLog)
+		})
+	}
 
 	t.Run("AttachMidStream", func(t *testing.T) {
 		t.Parallel()
@@ -433,7 +438,7 @@ func (tr Reader) TestConcurrent(t *testing.T) {
 	logAll := func(msgs []*logger.Message) {
 		defer wg.Done()
 		for _, m := range msgs {
-			l.Log(copyLogMessage(m))
+			assert.Check(t, l.Log(copyLogMessage(m)), "failed to log message %+v", m)
 		}
 	}
 
@@ -446,6 +451,15 @@ func (tr Reader) TestConcurrent(t *testing.T) {
 		defer l.Close()
 		wg.Wait()
 	}()
+	defer func() {
+		// Make sure log gets closed before we return
+		// so the temporary dir can be deleted
+		select {
+		case <-time.After(10 * time.Second):
+			t.Fatal("timed out waiting for logger to close")
+		case <-closed:
+		}
+	}()
 
 	// Check if the message count, order and content is equal to what was logged
 	for {
@@ -469,12 +483,8 @@ func (tr Reader) TestConcurrent(t *testing.T) {
 		*messages = (*messages)[1:]
 	}
 
-	assert.Equal(t, len(stdoutMessages), 0)
-	assert.Equal(t, len(stderrMessages), 0)
-
-	// Make sure log gets closed before we return
-	// so the temporary dir can be deleted
-	<-closed
+	assert.Check(t, is.Len(stdoutMessages, 0), "expected stdout messages were not read")
+	assert.Check(t, is.Len(stderrMessages, 0), "expected stderr messages were not read")
 }
 
 // logMessages logs messages to l and returns a slice of messages as would be