Repository: PrefectHQ/prefect

Issue:
`Local File System` block created on every flow run
### First check

- [X] I added a descriptive title to this issue.
- [X] I used the GitHub search to find a similar issue and didn't find it.
- [X] I searched the Prefect documentation for this issue.
- [X] I checked that this issue is related to Prefect and not one of its dependencies.

### Bug summary

When running a flow, a new block document is generated every time. Run the reproduction below and observe a new `LocalFileSystem` block document is generated each time.

### Reproduction

```python3
from prefect import flow

@flow(log_prints=True)
def my_flow():
    print(42)

if __name__ == '__main__':
    my_flow()
```


### Error

_No response_

### Versions

```Text
Version:             2.11.2
API version:         0.8.4
Python version:      3.10.9
Git commit:          eeab3d20
Built:               Tue, Aug 1, 2023 3:56 PM
OS/Arch:             darwin/x86_64
Profile:             postgresql
Server type:         ephemeral
Server:
  Database:          postgresql
```


### Additional context

Possibly the same/related to https://github.com/PrefectHQ/prefect/issues/9950
New storage blocks created on every flow run instead of using saved block
### First check

- [X] I added a descriptive title to this issue.
- [X] I used the GitHub search to find a similar issue and didn't find it.
- [X] I searched the Prefect documentation for this issue.
- [X] I checked that this issue is related to Prefect and not one of its dependencies.

### Bug summary

When using a saved storage block, a new block document is being generated for every flow run instead of referring to the block created prior. 

Issue created from conversation in Prefect Slack channel : https://prefect-community.slack.com/archives/CL09KU1K7/p1686212575760679

### Reproduction

```python3
import asyncio

import prefect
from prefect import flow, task
from prefect.filesystems import LocalFileSystem 
from prefect.serializers import JSONSerializer


# CREATES A NEW BLOCK IF IT DOES NOT EXIST
try:
    LocalFileSystem(basepath="./prefect_storage").save("a")
except ValueError:
    pass

# LOADS THE BLOCK
storage = LocalFileSystem.load("a")
# ENSURE THAT "_block_document_id" exists
assert "_block_document_id" in storage.__dict__


@task(result_storage=storage, persist_result=True, result_serializer=JSONSerializer())
async def task1():
    prefect.get_run_logger().info("Task run.")
    return "Success."


@flow
async def flow1():
    await task1()


if __name__ == "__main__":
    asyncio.run(flow1())
```


### Error

Running the code for the first time on a fresh server generates two blocks: 
1. Block named a [Expected]
2. An block created anonymously.

Every time, this flow is executed a new block is added to the block_document table.


### Versions

```Text
Version:             2.10.12
API version:         0.8.4
Python version:      3.10.6
Git commit:          ec55abdf
Built:               Thu, Jun 1, 2023 3:51 PM
OS/Arch:             linux/x86_64
Profile:             local_db
Server type:         server
```


### Additional context

We are having space issues on our database as we use flows that run large numbers of other deployments. We are generating over ~700MB of data (rough estimate) in this table alone on a daily basis.

For simplicity, Ive opted to using LocalFileSystem in the replication code. The isuue is the same with Azure block.


Discussion:

@OptimeeringBigya I believe the created block is actually for storing your _flow's_ results, so you might be able to workaround this behavior by doing:

```python
@flow(result_storage=storage) # same block as for tasks
async def flow1():
    await task1()
```

This is still undesirable behavior, so I'm going to look at ways to prevent this, but hopefully the workaround at least stops your DB from filling up!

Resolving patch:
diff --git a/src/prefect/client/orchestration.py b/src/prefect/client/orchestration.py
--- a/src/prefect/client/orchestration.py
+++ b/src/prefect/client/orchestration.py
@@ -1312,6 +1312,9 @@ async def read_block_document(
         Returns:
             A block document or None.
         """
+        assert (
+            block_document_id is not None
+        ), "Unexpected ID on block document. Was it persisted?"
         try:
             response = await self._client.get(
                 f"/block_documents/{block_document_id}",
diff --git a/src/prefect/results.py b/src/prefect/results.py
--- a/src/prefect/results.py
+++ b/src/prefect/results.py
@@ -118,7 +118,7 @@ class ResultFactory(pydantic.BaseModel):
     persist_result: bool
     cache_result_in_memory: bool
     serializer: Serializer
-    storage_block_id: uuid.UUID
+    storage_block_id: Optional[uuid.UUID]
     storage_block: WritableFileSystem
     storage_key_fn: Callable[[], str]
 
@@ -258,7 +258,7 @@ async def from_settings(
         client: "PrefectClient",
     ) -> Self:
         storage_block_id, storage_block = await cls.resolve_storage_block(
-            result_storage, client=client
+            result_storage, client=client, persist_result=persist_result
         )
         serializer = cls.resolve_serializer(result_serializer)
 
@@ -273,23 +273,31 @@ async def from_settings(
 
     @staticmethod
     async def resolve_storage_block(
-        result_storage: ResultStorage, client: "PrefectClient"
-    ) -> Tuple[uuid.UUID, WritableFileSystem]:
+        result_storage: ResultStorage,
+        client: "PrefectClient",
+        persist_result: bool = True,
+    ) -> Tuple[Optional[uuid.UUID], WritableFileSystem]:
         """
         Resolve one of the valid `ResultStorage` input types into a saved block
         document id and an instance of the block.
         """
         if isinstance(result_storage, Block):
             storage_block = result_storage
-            storage_block_id = (
+
+            if storage_block._block_document_id is not None:
                 # Avoid saving the block if it already has an identifier assigned
-                storage_block._block_document_id
-                # TODO: Overwrite is true to avoid issues where the save collides with
-                #       a previously saved document with a matching hash
-                or await storage_block._save(
-                    is_anonymous=True, overwrite=True, client=client
-                )
-            )
+                storage_block_id = storage_block._block_document_id
+            else:
+                if persist_result:
+                    # TODO: Overwrite is true to avoid issues where the save collides with
+                    # a previously saved document with a matching hash
+                    storage_block_id = await storage_block._save(
+                        is_anonymous=True, overwrite=True, client=client
+                    )
+                else:
+                    # a None-type UUID on unpersisted storage should not matter
+                    # since the ID is generated on the server
+                    storage_block_id = None
         elif isinstance(result_storage, str):
             storage_block = await Block.load(result_storage, client=client)
             storage_block_id = storage_block._block_document_id
@@ -486,6 +494,9 @@ async def get(self, client: "PrefectClient") -> R:
 
     @inject_client
     async def _read_blob(self, client: "PrefectClient") -> "PersistedResultBlob":
+        assert (
+            self.storage_block_id is not None
+        ), "Unexpected storage block ID. Was it persisted?"
         block_document = await client.read_block_document(self.storage_block_id)
         storage_block: ReadableFileSystem = Block._from_block_document(block_document)
         content = await storage_block.read_path(self.storage_key)
@@ -521,6 +532,10 @@ async def create(
         The object will be serialized and written to the storage block under a unique
         key. It will then be cached on the returned result.
         """
+        assert (
+            storage_block_id is not None
+        ), "Unexpected storage block ID. Was it persisted?"
+
         data = serializer.dumps(obj)
         blob = PersistedResultBlob(serializer=serializer, data=data)
 


Test patch:
diff --git a/tests/results/test_result_factory.py b/tests/results/test_result_factory.py
--- a/tests/results/test_result_factory.py
+++ b/tests/results/test_result_factory.py
@@ -2,6 +2,8 @@
 
 import pytest
 
+import prefect.exceptions
+import prefect.results
 from prefect import flow, task
 from prefect.context import get_run_context
 from prefect.filesystems import LocalFileSystem
@@ -59,7 +61,7 @@ def foo():
     assert result_factory.cache_result_in_memory is True
     assert result_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(result_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(result_factory.storage_block_id, uuid.UUID)
+    assert result_factory.storage_block_id is None
 
 
 def test_root_flow_default_result_serializer_can_be_overriden_by_setting():
@@ -103,7 +105,11 @@ def foo():
     assert result_factory.persist_result is toggle
     assert result_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(result_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(result_factory.storage_block_id, uuid.UUID)
+
+    if toggle:
+        assert isinstance(result_factory.storage_block_id, uuid.UUID)
+    else:
+        assert result_factory.storage_block_id is None
 
 
 @pytest.mark.parametrize("options", [{"cache_result_in_memory": False}])
@@ -132,7 +138,7 @@ def foo():
     assert result_factory.persist_result is False
     assert result_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(result_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(result_factory.storage_block_id, uuid.UUID)
+    assert result_factory.storage_block_id is None
 
 
 @pytest.mark.parametrize("toggle", [True, False])
@@ -149,7 +155,11 @@ def foo():
     assert result_factory.cache_result_in_memory is toggle
     assert result_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(result_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(result_factory.storage_block_id, uuid.UUID)
+
+    if toggle:
+        assert result_factory.storage_block_id is None
+    else:
+        assert isinstance(result_factory.storage_block_id, uuid.UUID)
 
 
 def test_root_flow_custom_serializer_by_type_string():
@@ -161,7 +171,7 @@ def foo():
     assert result_factory.persist_result is False
     assert result_factory.serializer == JSONSerializer()
     assert_blocks_equal(result_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(result_factory.storage_block_id, uuid.UUID)
+    assert result_factory.storage_block_id is None
 
 
 def test_root_flow_custom_serializer_by_instance():
@@ -173,7 +183,7 @@ def foo():
     assert result_factory.persist_result is False
     assert result_factory.serializer == JSONSerializer(jsonlib="orjson")
     assert_blocks_equal(result_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(result_factory.storage_block_id, uuid.UUID)
+    assert result_factory.storage_block_id is None
 
 
 def test_root_flow_custom_storage_by_slug(tmp_path):
@@ -210,12 +220,14 @@ def foo():
 async def test_root_flow_custom_storage_by_instance_unsaved(prefect_client, tmp_path):
     storage = LocalFileSystem(basepath=tmp_path)
 
-    @flow(result_storage=storage)
+    @flow(
+        result_storage=storage, cache_result_in_memory=False
+    )  # use a feature that requires persistence
     def foo():
         return get_run_context().result_factory
 
     result_factory = foo()
-    assert result_factory.persist_result is False
+    assert result_factory.persist_result is True
     assert result_factory.serializer == DEFAULT_SERIALIZER()
     assert result_factory.storage_block == storage
     assert result_factory.storage_block._is_anonymous is True
@@ -241,7 +253,7 @@ def bar():
     assert child_factory.persist_result is False
     assert child_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(child_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(child_factory.storage_block_id, uuid.UUID)
+    assert child_factory.storage_block_id is None
 
 
 def test_child_flow_default_result_serializer_can_be_overriden_by_setting():
@@ -326,7 +338,11 @@ def bar():
     assert child_factory.cache_result_in_memory is toggle
     assert child_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(child_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(child_factory.storage_block_id, uuid.UUID)
+
+    if toggle:
+        assert child_factory.storage_block_id is None
+    else:
+        assert isinstance(child_factory.storage_block_id, uuid.UUID)
 
 
 @pytest.mark.parametrize("options", [{"retries": 3}])
@@ -361,7 +377,7 @@ def bar():
     assert child_factory.persist_result is False
     assert child_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(child_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(child_factory.storage_block_id, uuid.UUID)
+    assert child_factory.storage_block_id is None
 
 
 @pytest.mark.parametrize("options", [{"cache_result_in_memory": False}])
@@ -401,7 +417,7 @@ def bar():
     assert child_factory.cache_result_in_memory is False
     assert child_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(child_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(child_factory.storage_block_id, uuid.UUID)
+    assert child_factory.storage_block_id is None
 
 
 def test_child_flow_inherits_custom_serializer():
@@ -417,7 +433,7 @@ def bar():
     assert child_factory.persist_result is False
     assert child_factory.serializer == parent_factory.serializer
     assert_blocks_equal(child_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(child_factory.storage_block_id, uuid.UUID)
+    assert child_factory.storage_block_id is None
 
 
 def test_child_flow_inherits_custom_storage(tmp_path):
@@ -453,7 +469,7 @@ def bar():
     assert child_factory.persist_result is False
     assert child_factory.serializer == JSONSerializer()
     assert_blocks_equal(child_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(child_factory.storage_block_id, uuid.UUID)
+    assert child_factory.storage_block_id is None
 
 
 def test_child_flow_custom_storage(tmp_path):
@@ -479,12 +495,14 @@ def bar():
 async def test_child_flow_custom_storage_by_instance_unsaved(prefect_client, tmp_path):
     storage = LocalFileSystem(basepath=tmp_path)
 
-    @flow()
+    @flow(cache_result_in_memory=False)  # use a feature that requires persistence
     def foo():
+        print(f"In parent, persist={get_run_context().result_factory.persist_result}")
         return get_run_context().result_factory, bar()
 
-    @flow(result_storage=storage)
+    @flow(result_storage=storage, cache_result_in_memory=False)
     def bar():
+        print(f"In child, persist={get_run_context().result_factory.persist_result}")
         return get_run_context().result_factory
 
     parent_factory, child_factory = foo()
@@ -505,7 +523,7 @@ def bar():
     assert LocalFileSystem._from_block_document(storage_block_document) == storage
 
     # Other settings should not be changed
-    assert child_factory.persist_result is False
+    assert child_factory.persist_result is True
     assert child_factory.serializer == DEFAULT_SERIALIZER()
 
 
@@ -522,7 +540,7 @@ def bar():
     assert task_factory.persist_result is False
     assert task_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(task_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(task_factory.storage_block_id, uuid.UUID)
+    assert task_factory.storage_block_id is None
 
 
 def test_task_default_result_serializer_can_be_overriden_by_setting():
@@ -592,7 +610,11 @@ def bar():
     assert task_factory.cache_result_in_memory is toggle
     assert task_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(task_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(task_factory.storage_block_id, uuid.UUID)
+
+    if toggle:
+        assert task_factory.storage_block_id is None
+    else:
+        assert isinstance(task_factory.storage_block_id, uuid.UUID)
 
 
 @pytest.mark.parametrize("options", [{"retries": 3}])
@@ -647,7 +669,7 @@ def bar():
     assert task_factory.persist_result is False
     assert task_factory.serializer == DEFAULT_SERIALIZER()
     assert_blocks_equal(task_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(task_factory.storage_block_id, uuid.UUID)
+    assert task_factory.storage_block_id is None
 
 
 def test_task_can_opt_out_when_persist_result_default_is_overriden_by_setting():
@@ -678,7 +700,7 @@ def bar():
     assert task_factory.persist_result is False
     assert task_factory.serializer == flow_factory.serializer
     assert_blocks_equal(task_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(task_factory.storage_block_id, uuid.UUID)
+    assert task_factory.storage_block_id is None
 
 
 def test_task_inherits_custom_storage(tmp_path):
@@ -714,7 +736,7 @@ def bar():
     assert task_factory.persist_result is False
     assert task_factory.serializer == JSONSerializer()
     assert_blocks_equal(task_factory.storage_block, DEFAULT_STORAGE())
-    assert isinstance(task_factory.storage_block_id, uuid.UUID)
+    assert task_factory.storage_block_id is None
 
 
 def test_task_custom_storage(tmp_path):
@@ -740,11 +762,11 @@ def bar():
 async def test_task_custom_storage_by_instance_unsaved(prefect_client, tmp_path):
     storage = LocalFileSystem(basepath=tmp_path)
 
-    @flow()
+    @flow(cache_result_in_memory=False)
     def foo():
         return get_run_context().result_factory, bar()
 
-    @flow(result_storage=storage)
+    @flow(result_storage=storage, cache_result_in_memory=False)
     def bar():
         return get_run_context().result_factory
 
@@ -766,5 +788,107 @@ def bar():
     assert LocalFileSystem._from_block_document(storage_block_document) == storage
 
     # Other settings should not be changed
-    assert task_factory.persist_result is False
+    assert task_factory.persist_result is True
     assert task_factory.serializer == DEFAULT_SERIALIZER()
+
+
+async def _verify_default_storage_creation_with_persistence(
+    prefect_client,
+    result_factory: prefect.results.ResultFactory,
+):
+    # check that the default block was created
+    assert result_factory.storage_block is not None
+    assert_blocks_equal(result_factory.storage_block, DEFAULT_STORAGE())
+
+    # verify storage settings are correctly set
+    assert result_factory.persist_result is True
+    assert isinstance(result_factory.storage_block_id, uuid.UUID)
+
+    # verify the remote block exists
+    storage_block_document = await prefect_client.read_block_document(
+        result_factory.storage_block_id
+    )
+    # unnecessary since the read_block_document call above will fail if the
+    # block doesn't exist
+    assert storage_block_document
+
+
+async def _verify_default_storage_creation_without_persistence(
+    result_factory: prefect.results.ResultFactory,
+):
+    # check that the default block was created
+    assert result_factory.storage_block is not None
+    assert_blocks_equal(result_factory.storage_block, DEFAULT_STORAGE())
+
+    # verify storage settings are correctly set
+    assert result_factory.storage_block._is_anonymous is None
+    assert result_factory.persist_result is False
+    assert result_factory.storage_block_id is None
+
+
+@pytest.mark.parametrize(
+    "options", [{"persist_result": True}, {"cache_result_in_memory": False}]
+)
+async def test_default_storage_creation_for_flow_with_persistence_features(
+    prefect_client, options
+):
+    @flow(**options)
+    def foo():
+        return get_run_context().result_factory
+
+    result_factory = foo()
+    await _verify_default_storage_creation_with_persistence(
+        prefect_client, result_factory
+    )
+
+
+async def test_default_storage_creation_for_flow_without_persistence_features():
+    @flow()
+    def foo():
+        return get_run_context().result_factory
+
+    result_factory = foo()
+    await _verify_default_storage_creation_without_persistence(result_factory)
+
+
+async def test_default_storage_creation_for_task_with_persistence_features(
+    prefect_client,
+):
+    @task
+    def my_task_1():
+        return get_run_context().result_factory
+
+    @flow(retries=2)
+    def my_flow_1():
+        return my_task_1()
+
+    result_factory = my_flow_1()
+    await _verify_default_storage_creation_with_persistence(
+        prefect_client, result_factory
+    )
+
+    @task(cache_key_fn=lambda *_: "always")
+    def my_task_2():
+        return get_run_context().result_factory
+
+    @flow
+    def my_flow_2():
+        return my_task_2()
+
+    result_factory = my_flow_2()
+    await _verify_default_storage_creation_with_persistence(
+        prefect_client, result_factory
+    )
+
+
+async def test_default_storage_creation_for_task_without_persistence_features():
+    @task
+    def my_task():
+        return get_run_context().result_factory
+
+    @flow()
+    def my_flow():
+        return my_task()
+
+    result_factory = my_flow()
+    await _verify_default_storage_creation_without_persistence(result_factory)
diff --git a/tests/test_states.py b/tests/test_states.py
--- a/tests/test_states.py
+++ b/tests/test_states.py
@@ -160,8 +160,10 @@ async def test_returns_single_state_with_null_data(self, factory):
         assert isinstance(result_state.data, UnpersistedResult)
         assert await result_state.result() is None
 
-    async def test_returns_single_state_with_data_to_persist(self, factory):
-        factory.persist_result = True
+    async def test_returns_single_state_with_data_to_persist(self, prefect_client):
+        factory = await ResultFactory.default_factory(
+            client=prefect_client, persist_result=True
+        )
         state = Completed(data=1)
         result_state = await return_value_to_state(state, factory)
         assert result_state is state