Change:
<!-- Thank you for filing a bug report! Please provide a short summary of the bug, along with any information you feel relevant to replicating the bug. --Running Fedora 32 with selinux set to non-enforcing, building a project with OpenCV. I am building a rather weird docker setup, but what I ran into was hanging during build stage of opencv-rust, . I expected to see this happen: build succeed rather quickly Instead, this happened: execution froze on , and cpu got stuck running almost full speed. My project was mounted using (name changed), opencv was manually cloned (for debugging) into the root of the docker image, at . Copies from to don't work (example: =) File copies from to work just fine (example: =) It appears to be an issue in the implementation of , as implementing does not freeze the operation. <!-- If you're using the stable version of the compiler, you should also check if the bug also exists in the beta or nightly versions. --: </p</details<!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
Update: I reimplemented the Linux version, and the count always gets stuck on in the function.
Any chance you can the process that gets stuck?
It's an endless
According to man, , although the behaviour is odd. If I were to guess running this under docker does not return , even though it is technically on external device (not in the view of the docker container though).
This particular case of 0 is not being handled in , only the errors. Adding match arms and redirecting 0 to one of the errors to force using fallback mode could work. However, there may be cases where 0 gets returned legitimately?
The current implementation queries the size of the source file and then does a decrementing loop. Does the file in question get truncated by another thread while the copy is in progress?
Can you check which syscalls are used in the direction that works? Recent kernels support cross device copyfilerange under certain circumstances, e.g. overlayfs can delegate to the underlying device and btrfs can copy between volumes. Bind mounts also should count as the same device. But maybe there's a bug in there in combination with selinux?
If selinux is non-enforcing (per OP), that should rule it out.
It still seems really odd. The kernel's copyfilerange implementation itself does a lot of fallbacks internally when the preferred methods return a 0, i.e. 0 bytes copied ultimately lead to a splice operation from file to file which is not all that different than what we're doing in userspace. So 0 bytes being copied is quite unexpected for a non-empty file. There are lots of layers involved though and any of them could be the cause. Adding yet another workaround is possible, but if possible I'd like to identify the root cause so we can report it to the responsible parties.
Coreutils tests for a zero-byte copyfilerange() and falls back on a read()/write() loop if that happens: Their code has a comment mentioning that this happens for /proc special files. I tried to reproduce it with but ran into another bug: stat() says that file is empty, so fs::copy() creates an empty file and doesn't copy anything. I think trusting st_size is fundamentally broken, for both this reason and the race that mentioned.
claim
that's another good reason to change the logic, but it'd still be good to know what's actually causing it so we can add the root cause to comments or report things upstream if they haven't been fixed already. The direction-dependence smells like something overlayfs-related, but that's just a guess without further details from
This is interesting. Kernel's sets the length to zero upon return, because returns 0. Which ultimately means, that the inode's size in the kernel is zero, while the metadata returns size of . Possible kernel bug? Yet, this behaviour only occurs when copying from docker's system over to overlayed home directory, all technically on the same overlayfs, but on different filesystems on the host side.
Maybe it's getting from the stacked inode in the overlayfs, not the underlying one
I'm trying to reproduce it from basic pieces but so far it works just fine kernel 5.7.12-arch1-1 btrfs
I'm sorry I made one horrible mistake, the directories are not mounted on the same overlayfs, they seem to be bind mounted Output on docker container: I am not sure how exactly would one reproduce this environment without actually using docker/podman to do so.
Not only that, I noticed that the behaviour is very strange. Some files do cause the lock ups, some don't. Now, building by using directly from , running build script creates this file: Running fs::copy script with the path causes it to lock up. But, if I cp the file elsewhere, and copy it back in place, fs::copy does not cause any more lockups. If I append a new line to the file, no more lock ups are caused, so much so as changing file permissions fixes it. Basically, if I modify the file in any shape or form, it seems to update itself and work just fine. This really seems like some serious, hard to reproduce kernel bug to me.
My current kernel version is , but the same happened with an older 5.7 kernel as well.
Those are the target directories. But the source also matters and depending on the docker storage driver you're using that might be overlayfs. Yeah, again if overlayfs is involved that may make a difference whether the file comes from the upper or the lower. A step by step reduced testcase would help. I grasp the rough outline what is happening but there are many details that might make a difference. I can add a workaround without that, but then I can't verify that the issue is fixed and it would make reporting things upstream more difficult too.

Discussion:
} let (mut reader, reader_metadata) = open_from(from)?; <del> let len = reader_metadata.len(); </del> <ins> let max_len = u64::MAX; </ins> let (mut writer, _) = open_to_and_set_permissions(to, reader_metadata)?; let has_copy_file_range = HAS_COPY_FILE_RANGE.load(Ordering::Relaxed); let mut written = 0u64; <del> while written < len { </del> <ins> while written < max_len { </ins> let copy_result = if has_copy_file_range { <del> let bytes_to_copy = cmp::min(len - written, usize::MAX as u64) as usize; </del> <ins> let bytes_to_copy = cmp::min(max_len - written, usize::MAX as u64) as usize; </ins> let copy_result = unsafe { // We actually don't have to adjust the offsets, // because copy_file_range adjusts the file offset automatically

}; if let Err(ref copy_err) = copy_result { match copy_err.raw_os_error() { <del> Some(libc::ENOSYS) | Some(libc::EPERM) => { </del> <ins> Some(libc::ENOSYS | libc::EPERM | libc::EOPNOTSUPP) => { </ins> HAS_COPY_FILE_RANGE.store(false, Ordering::Relaxed); } _ => {}

Err(io::Error::from_raw_os_error(libc::ENOSYS)) }; match copy_result { <ins> Ok(0) if written == 0 => { // fallback to work around several kernel bugs where copy_file_range will fail to // copy any bytes and return 0 instead of an error if // - reading virtual files from the proc filesystem which appear to have 0 size //   but are not empty. noted in coreutils to affect kernels at least up to 5.6.19. // - copying from an overlay filesystem in docker. reported to occur on fedora 32. return io::copy(&mut reader, &mut writer); } Ok(0) => return Ok(written), // reached EOF </ins> Ok(ret) => written += ret as u64, Err(err) => { match err.raw_os_error() { <del> Some(os_err) if os_err == libc::ENOSYS || os_err == libc::EXDEV || os_err == libc::EINVAL || os_err == libc::EPERM => { </del> <ins> Some( libc::ENOSYS | libc::EXDEV | libc::EINVAL | libc::EPERM | libc::EOPNOTSUPP, ) => { </ins> // Try fallback io::copy if either: // - Kernel version is < 4.5 (ENOSYS) // - Files are mounted on different fs (EXDEV) <ins> // - copy_file_range is broken in various ways on RHEL/CentOS 7 (EOPNOTSUPP) </ins> // - copy_file_range is disallowed, for example by seccomp (EPERM) // - copy_file_range cannot be used with pipes or device nodes (EINVAL) assert_eq!(written, 0);