Repository: checkpoint-restore/criu (C)

Issue:
Does criu not restore the read-only attribute of the container's rootfs?
I use Docker to start a container, ensuring the container's rootfs is read-only by specifying the `--read-only` flag. Then, I save and resume the container using Docker's integrated checkpoint command. After restoring the container, when I exec into it, I notice that the rootfs has become writable. Here are the simple steps to reproduce the issue:

```
# docker run -td --read-only ubuntu:22.04 sleep inf
8cd6084fc2aa2584443aa5d2151ae7884b6c4b889a8bd869545255c1856574a6

# docker exec -ti 8cd6084fc2aa2584443aa5d2151ae7884b6c4b889a8bd869545255c1856574a6 bash
bash-4.4#
bash-4.4# findmnt /
TARGET SOURCE  FSTYPE  OPTIONS
/      overlay overlay ro,relatime,lowerdir=/var/lib/docker/overlay2/l/LEQGJFKK7AP52BWBHFYWUTSY7H:/var/lib/docker/....
bash-4.4#
```

The root directory of the container can be seen as read-only at this moment. Next, we will perform checkpoint/restore on this container:
```
# docker checkpoint create 8cd6084fc2aa2584443aa5d2151ae7884b6c4b889a8bd869545255c1856574a6 checkpoint1
checkpoint1

# docker start --checkpoint checkpoint1 8cd6084fc2aa2584443aa5d2151ae7884b6c4b889a8bd869545255c1856574a6
# docker exec -ti 8cd6084fc2aa2584443aa5d2151ae7884b6c4b889a8bd869545255c1856574a6 bash
bash-4.4#
bash-4.4# findmnt  /
TARGET SOURCE  FSTYPE  OPTIONS
/      overlay overlay rw,relatime,lowerdir=/var/lib/docker/overlay2/l/LEQGJFKK7AP52BWBHFYWUTSY7H:/var/lib/docker/...
```

At this point, the container's rootfs attribute has become rw.






Discussion: That sounds like a docker problem. The rootfs is pretty sure mounted by docker and then CRIU is called.
@adrianreber So CRIU doesn't save and restore any mount attributes for the container's rootfs, right?
No, only mounts on top of the rootfs
Understood, thank you.

I have one more question. If the dumped process has a mnt namespace, it seems from the code that CRIU will always call unshare() to create a new mnt namespace, even though I specified the `--join-ns mnt:pid` option. It appears this option is not having the intended effect. By the way, I noticed that at this link: https://github.com/checkpoint-restore/criu/blob/criu-dev/Documentation/criu.txt#L542, there's a list of supported and unsupported cases for join-ns, but mnt namespace is not mentioned at all. Is it not supported?
> That sounds like a docker problem. The rootfs is pretty sure mounted by docker and then CRIU is called.

I think runc handles whether a container roofs will be ro or rw in a container: https://github.com/opencontainers/runc/blob/main/libcontainer/rootfs_linux.go#L251


I think the next patch has to fix the issue:
```
diff --git a/libcontainer/criu_linux.go b/libcontainer/criu_linux.go
index 59befe45..9537d276 100644
--- a/libcontainer/criu_linux.go
+++ b/libcontainer/criu_linux.go
@@ -617,6 +617,21 @@ func (c *Container) prepareCriuRestoreMounts(mounts []*configs.Mount) error {
        return nil
 }
 
+func setMountReadonly(dst string) error {
+       flags := uintptr(unix.MS_BIND | unix.MS_REMOUNT | unix.MS_RDONLY)
+
+       err := mount("", dst, "", flags, "")
+       if err == nil {
+               return nil
+       }
+       var s unix.Statfs_t
+       if err := unix.Statfs(dst, &s); err != nil {
+               return &os.PathError{Op: "statfs", Path: "/", Err: err}
+       }
+       flags |= uintptr(s.Flags)
+       return mount("", dst, "", flags, "")
+}
+
 // Restore restores the checkpointed container to a running state using the
 // criu(8) utility.
 func (c *Container) Restore(process *Process, criuOpts *CriuOpts) error {
@@ -668,6 +683,11 @@ func (c *Container) Restore(process *Process, criuOpts *CriuOpts) error {
        if err != nil {
                return err
        }
+       if c.config.Readonlyfs {
+               if err := setMountReadonly(root); err != nil {
+                       return err
+               }
+       }
        defer unix.Unmount(root, unix.MNT_DETACH) //nolint: errcheck
        t := criurpc.CriuReqType_RESTORE
        req := &criurpc.CriuReq{
```

It is still a question whether CRIU has to restore bind-mount flags for external mounts. @Snorch what do you think?
> It is still a question whether CRIU has to restore bind-mount flags for external mounts. @Snorch what do you think?

I think CRIU should restore mount flags. It probably should look like this:
```
diff --git a/criu/mount-v2.c b/criu/mount-v2.c
index 5d53e9a22..b9eb66072 100644
--- a/criu/mount-v2.c
+++ b/criu/mount-v2.c
@@ -443,6 +443,7 @@ err:
 /* Mounts root container mount. */
 static int do_mount_root_v2(struct mount_info *mi)
 {
+       unsigned long mflags = mi->flags & (~MS_PROPAGATE);
        unsigned long flags = MS_BIND;
        int fd;
 
@@ -477,6 +478,11 @@ static int do_mount_root_v2(struct mount_info *mi)
                return -1;
        }
 
+       if (mflags && mount(NULL, mi->plain_mountpoint, NULL, MS_REMOUNT | MS_BIND | mflags, NULL)) {
+               pr_perror("Unable to apply bind-mount options");
+               return -1;
+       }
+
        mi->mounted = true;
 
        return 0;
```
>> It is still a question whether CRIU has to restore bind-mount flags for external mounts. @Snorch what do you think?
>
> I think CRIU should restore mount flags. It probably should look like this:

Yes, I agree, CRIU must restore mount flags as they belong to the mount and thus to thecontainer mount namespace. The patch looks good.



Fix (patch):
diff --git a/criu/mount-v2.c b/criu/mount-v2.c
index 5d53e9a226..eb4dd8119a 100644
--- a/criu/mount-v2.c
+++ b/criu/mount-v2.c
@@ -443,6 +443,7 @@ static int do_bind_mount_v2(struct mount_info *mi)
 /* Mounts root container mount. */
 static int do_mount_root_v2(struct mount_info *mi)
 {
+	unsigned long mflags = mi->flags & (~MS_PROPAGATE);
 	unsigned long flags = MS_BIND;
 	int fd;
 
@@ -477,6 +478,11 @@ static int do_mount_root_v2(struct mount_info *mi)
 		return -1;
 	}
 
+	if (mflags && mount(NULL, mi->plain_mountpoint, NULL, MS_REMOUNT | MS_BIND | mflags, NULL)) {
+		pr_perror("Unable to apply root mount options");
+		return -1;
+	}
+
 	mi->mounted = true;
 
 	return 0;
diff --git a/criu/mount.c b/criu/mount.c
index 82bbd52d6c..06b9595427 100644
--- a/criu/mount.c
+++ b/criu/mount.c
@@ -2690,9 +2690,16 @@ static bool can_mount_now(struct mount_info *mi)
 
 static int do_mount_root(struct mount_info *mi)
 {
+	unsigned long mflags = mi->flags & (~MS_PROPAGATE);
+
 	if (restore_shared_options(mi, !mi->shared_id && !mi->master_id, mi->shared_id, mi->master_id))
 		return -1;
 
+	if (mflags && mount(NULL, service_mountpoint(mi), NULL, MS_REMOUNT | MS_BIND | mflags, NULL)) {
+		pr_perror("Unable to apply root mount options");
+		return -1;
+	}
+
 	return fetch_rt_stat(mi, service_mountpoint(mi));
 }