Repository: huggingface/diffusers
Issue category: Bug Report

Issue:
Replace shutil.copy with shutil.copyfile
shutil.copy copies permission bits which fails when the user who's running the script is trying to use a common cache that was generated by another user, even though the first user has read & write permissions over the cache (through Group permission for example). A real case scenario: submitting jobs on a GPU cluster accessed by multiple users with common cache directory to reduce disk and network usage.

https://github.com/huggingface/diffusers/blob/0d1d267b12e47b40b0e8f265339c76e0f45f8c49/src/diffusers/utils/dynamic_modules_utils.py#L328

Suggested solution: replace shutil.copy by shutil.copyfile which doesn't copy ownership.


Hints:
Maybe related to https://github.com/huggingface/huggingface_hub/pull/1220, https://github.com/huggingface/diffusers/issues/1517, https://github.com/huggingface/huggingface_hub/issues/1141.
For info I'm using v0.23.0.
@Wauplin WDYT? 
Agree with using `shutil.copyfile` yes! I didn't thought about permission issues back then. copyfile will only copy the file content but that is what we want here :) 
 
@almarouk would you be willing to open a PR to help us?

Patch:
diff --git a/src/diffusers/utils/dynamic_modules_utils.py b/src/diffusers/utils/dynamic_modules_utils.py
index f0cf953924ad..50d9bbaac57c 100644
--- a/src/diffusers/utils/dynamic_modules_utils.py
+++ b/src/diffusers/utils/dynamic_modules_utils.py
@@ -325,7 +325,7 @@ def get_cached_module_file(
         # We always copy local files (we could hash the file to see if there was a change, and give them the name of
         # that hash, to only copy when there is a modification but it seems overkill for now).
         # The only reason we do the copy is to avoid putting too many folders in sys.path.
-        shutil.copy(resolved_module_file, submodule_path / module_file)
+        shutil.copyfile(resolved_module_file, submodule_path / module_file)
         for module_needed in modules_needed:
             if len(module_needed.split(".")) == 2:
                 module_needed = "/".join(module_needed.split("."))
@@ -333,7 +333,7 @@ def get_cached_module_file(
                 if not os.path.exists(submodule_path / module_folder):
                     os.makedirs(submodule_path / module_folder)
             module_needed = f"{module_needed}.py"
-            shutil.copy(os.path.join(pretrained_model_name_or_path, module_needed), submodule_path / module_needed)
+            shutil.copyfile(os.path.join(pretrained_model_name_or_path, module_needed), submodule_path / module_needed)
     else:
         # Get the commit hash
         # TODO: we will get this info in the etag soon, so retrieve it from there and not here.
@@ -350,7 +350,7 @@ def get_cached_module_file(
                 module_folder = module_file.split("/")[0]
                 if not os.path.exists(submodule_path / module_folder):
                     os.makedirs(submodule_path / module_folder)
-            shutil.copy(resolved_module_file, submodule_path / module_file)
+            shutil.copyfile(resolved_module_file, submodule_path / module_file)
 
         # Make sure we also have every file with relative
         for module_needed in modules_needed:


Test patch: