Repository: PrefectHQ/prefect

Issue:
New Docker Container with imbedded flows throws error when loading while parsing directory
### First check

- [X] I added a descriptive title to this issue.
- [X] I used the GitHub search to find a similar issue and didn't find it.
- [X] I searched the Prefect documentation for this issue.
- [X] I checked that this issue is related to Prefect and not one of its dependencies.

### Bug summary

When using the new method of building a deployment and including -ib and no -sb (no external storage block, everything is in the image), the deployment builds and applies fine.

When you run the deployment, it pulls the requested image correctly, and if the flows are NOT in the container or the directory it's looking for does not exist (whether you are using the default path or the --path override) it throws an appropriate error.

When you run the deployment and the flows ARE in the container and directory, it runs but throws an shutil error trying to copy something (not sure why it's trying to copy if the flow is already in the image anyway). It seems to be parsing the directory and error, but it does show the files that would be expected to be in that directory.

It also appears to create a new Docker Container block for each run when you provide an image override. You can see in the details below that it mentions ALL of the blocks each time, though only seems to actually run the latest?

### Reproduction

```python
12:04:49.408 | INFO    | prefect.infrastructure.docker-container - Docker container 'Test-Block-3' has status 'exited'
12:19:02.313 | INFO    | prefect.agent - Submitting flow run '96652a9a-bcfc-414c-9f3f-c479351de0a1'
12:19:02.638 | INFO    | prefect.infrastructure.docker-container - Pulling image 'docker.io/scoussens/app-sync-dev:latest'...
12:19:19.336 | INFO    | prefect.infrastructure.docker-container - Creating Docker container 'Test-Block'...
12:19:19.354 | INFO    | prefect.infrastructure.docker-container - Creating Docker container 'Test-Block-1'...
12:19:19.364 | INFO    | prefect.infrastructure.docker-container - Creating Docker container 'Test-Block-2'...
12:19:19.375 | INFO    | prefect.infrastructure.docker-container - Creating Docker container 'Test-Block-3'...
12:19:19.386 | INFO    | prefect.infrastructure.docker-container - Creating Docker container 'Test-Block-4'...
12:19:20.304 | INFO    | prefect.agent - Completed submission of flow run '96652a9a-bcfc-414c-9f3f-c479351de0a1'
12:19:20.322 | INFO    | prefect.infrastructure.docker-container - Docker container 'Test-Block-4' has status 'running'
17:19:22.783 | ERROR   | Flow run 'fresh-dogfish' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "/root/.local/lib/python3.9/site-packages/prefect/engine.py", line 256, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/root/.local/lib/python3.9/site-packages/prefect/client.py", line 103, in with_injected_client
    return await fn(*args, **kwargs)
  File "/root/.local/lib/python3.9/site-packages/prefect/deployments.py", line 54, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "/root/.local/lib/python3.9/site-packages/prefect/filesystems.py", line 134, in get_directory
    shutil.copytree(from_path, local_path, dirs_exist_ok=True)
  File "/usr/local/lib/python3.9/shutil.py", line 568, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
  File "/usr/local/lib/python3.9/shutil.py", line 522, in _copytree
    raise Error(errors)
shutil.Error: ['<', 'D', 'i', 'r', 'E', 'n', 't', 'r', 'y', ' ', "'", 'l', 'i', 'b', 'r', 'a', 'r', 'y', '-', '0', '.', '1', '.', '0', {truncated for brevity}
```


### Error

```
Traceback (most recent call last):
  File "/root/.local/lib/python3.9/site-packages/prefect/engine.py", line 256, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/root/.local/lib/python3.9/site-packages/prefect/client.py", line 103, in with_injected_client
    return await fn(*args, **kwargs)
  File "/root/.local/lib/python3.9/site-packages/prefect/deployments.py", line 54, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "/root/.local/lib/python3.9/site-packages/prefect/filesystems.py", line 134, in get_directory
    shutil.copytree(from_path, local_path, dirs_exist_ok=True)
  File "/usr/local/lib/python3.9/shutil.py", line 568, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
  File "/usr/local/lib/python3.9/shutil.py", line 522, in _copytree
    raise Error(errors)
shutil.Error: ['<', 'D', 'i', 'r', 'E', 'n', 't', 'r', 'y', ' ', "'", 'l', 'i', 'b', 'r', 'a', 'r', 'y', '-', '0', '.', '1', '.', '0'
```

This is using the default /opt/prefect/flows working directory for the entrypoint, but it does the same if we put the files in a different directory and use the --path override.


### Versions

```

Version:             2.3.0
API version:         0.8.0
Python version:      3.9.10
Git commit:          8d9316c0
Built:               Tue, Aug 30, 2022 5:30 PM
OS/Arch:             linux/x86_64
Profile:             scoussens-laptop
Server type:         cloud

```


### Additional context

The intention here is to put all of the flow code and packages into a Docker image, and then upload to a repository. This works and the image is pulled.

We have this working with azure storage currently, but that's really just an extra step as we already build the container with the flow code + packages built in. The azure step is entirely unnecessary as far as the actual code goes.

This is using the new functionality that was provided in v2.3.0


Discussion:
Quick update, the 'Creating container' duplicate entries appears to be logging related. It's seeing that there is already a local container with the name, so appending a new # to the end when creating the new container.

The problem I see with this is that instead of using the existing container if nothing has changed (hash?) it creates a new container for the next run. It's possible this is required, as my knowledge of docker is novice at best. Seems it would not be though since we are just running the same code with new parameters.
Hi @scoussens would you mind sharing with me the following info so that I can investigate deeper:
- your deployment specification (e.g., your deployment YAML file)
- the OS information from your docker image
I'll try and get you the details ASAP. I've moved to another part of the
upgrade for now, but will revisit this later this week.

Seth


On Sat, Sep 3, 2022 at 6:20 PM Chris White ***@***.***> wrote:

> Hi @scoussens <https://github.com/scoussens> would you mind sharing with
> me the following info so that I can investigate deeper:
>
>    - your deployment specification (e.g., your deployment YAML file)
>    - the OS information from your docker image
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/PrefectHQ/prefect/issues/6652#issuecomment-1236212336>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB5DSK6ZRPUYWXYCOYKACILV4PMNLANCNFSM6AAAAAAQBSYNEM>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>

I'm facing the same issue. The root cause is that, when running inside a container that already the flow as part of the build process, or mounted into the filesystem, the code in `prefect/deployments.py:load_flow_from_flow_run` still defaults to copying the code into a temporary directory.

If that directory does not exists (ie. you didn't create `/opt/prefect` during your image build process **and set it as the WORKDIR**), things break.

My workaround for now was to create the `/opt/prefect` folder and set `chdir` to it in a custom entrypoint. 

IMO, there should be a way to configure the working folder instead of assuming '.', and `LocalFileSystem` could be extended to provide symlinking instead of coping as a cheaper alternative 
@cjalmeida I think you may be describing something different - the _default_ location that Prefect looks for flows is `/opt/prefect/flows` but you can override that by setting the deployment's path to an absolute directory path ([as in this example](https://medium.com/the-prefect-blog/prefect-2-3-0-adds-support-for-flows-defined-in-docker-images-and-github-repositories-79a8797a7371#6413)) without needing `/opt/prefect` at all.

Agreed that symlinking is an interesting idea! 

Resolving patch:
diff --git a/src/prefect/filesystems.py b/src/prefect/filesystems.py
--- a/src/prefect/filesystems.py
+++ b/src/prefect/filesystems.py
@@ -123,10 +123,19 @@ async def get_directory(
         Defaults to copying the entire contents of the block's basepath to the current working directory.
         """
         if from_path is None:
-            from_path = Path(self.basepath).expanduser()
+            from_path = Path(self.basepath).expanduser().resolve()
+        else:
+            from_path = Path(from_path).resolve()
 
         if local_path is None:
-            local_path = Path(".").absolute()
+            local_path = Path(".").resolve()
+        else:
+            local_path = Path(local_path).resolve()
+
+        if from_path == local_path:
+            # If the paths are the same there is no need to copy
+            # and we avoid shutil.copytree raising an error
+            return
 
         if sys.version_info < (3, 8):
             shutil.copytree(from_path, local_path)


Test patch:
diff --git a/tests/test_filesystems.py b/tests/test_filesystems.py
--- a/tests/test_filesystems.py
+++ b/tests/test_filesystems.py
@@ -40,6 +40,10 @@ async def test_resolve_path(self, tmp_path):
         assert fs._resolve_path(tmp_path / "subdirectory") == tmp_path / "subdirectory"
         assert fs._resolve_path("subdirectory") == tmp_path / "subdirectory"
 
+    async def test_get_directory_duplicate_directory(self, tmp_path):
+        fs = LocalFileSystem(basepath=str(tmp_path))
+        await fs.get_directory(".", ".")
+
 
 class TestRemoteFileSystem:
     def test_must_contain_scheme(self):