Repository: dagster-io/dagster (Python)

Issue:
"ContextVar was created in a different Context" error from an async asset after successful materialization
### Dagster version

1.7.10

### What's the issue?

After upgrading to 1.7.10 I got this error from an async asset:

```
ValueError: <Token var=<ContextVar name='_warnings_on' default=True at 0x7f1b7fb13010> at 0x7f1b5791bc40> was created in a different Context
```
The issue seems to have been introduced in 1.7.8. With 1.7.7 it works as expected. Neither 1.7.9 or 1.7.10 works.

I can't really tie this to anything in my code. The asset doesn't even use an `AssetExecutionContext` (I don't even know if `ContextVar` has anything to do with it). Strangely enough the error pops up _after_ a green `STEP_OUTPUT` and an `ASSET_MATERIALIZATION` and a `HANDLED_OUTPUT` event.

![image](https://github.com/dagster-io/dagster/assets/12885402/bf56a202-f2bd-42eb-bb35-e8ff896f32c6)


### What did you expect to happen?

I expected my good ol' code to remain operational after upgrading just as before.

### How to reproduce?

Can't share the code, but it's been working for ages before upgrading. I have a very simple `async` asset function and an async context manager resource that wraps a REST API and also a db resource that is a connection to a Postgres db. 

The async context manager is essentially a `aiohttp.ClientSession` and I use that to fetch data from the API. The asset returns a list of Pydantic objects wrapped in an `Output`.

**Update:** the culprit seems to be the usage of `Output` at the end of the asset function. 

Code very similar to this fails:
``` python
# ... do stuff
return Output(results, metadata={"some": "metadata"})
```

but this works:

```python
# ... do stuff
context.add_output_metadata({"some": "metadata"})
return results
```

### Deployment type

Docker Compose

### Deployment details

multi container docker compose

### Additional information

Full error message:

```
ValueError: <Token var=<ContextVar name='_warnings_on' default=True at 0x7f1b7fb13010> at 0x7f1b5791bc40> was created in a different Context
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 54, in op_execution_error_boundary
    yield
  File "/usr/local/lib/python3.10/site-packages/dagster/_utils/__init__.py", line 468, in iterate_with_context
    next_output = next(iterator)
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/compute.py", line 131, in gen_from_async_gen
    yield loop.run_until_complete(async_gen.__anext__())
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/compute_generator.py", line 103, in _coerce_async_op_to_async_gen
    for event in validate_and_coerce_op_result_to_iterator(result, context, output_defs):
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/compute_generator.py", line 353, in validate_and_coerce_op_result_to_iterator
    with disable_dagster_warnings():
  File "/usr/local/lib/python3.10/contextlib.py", line 142, in __exit__
    next(self.gen)
  File "/usr/local/lib/python3.10/site-packages/dagster/_utils/warnings.py", line 146, in disable_dagster_warnings
    _warnings_on.reset(token)
```
### Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.


Discussion:
Updated the repro section, using `Output` seems to be the cause.
Hi there. I think the PR that introduced this issue is https://github.com/dagster-io/dagster/pull/22070 . we're investigating a fix
@alangenfeld I still have this same issue in dagster==1.7.13 and this works in 1.7.9

My Op:

```
@op
async def start_tag_endpoints(context):
    results = []
    async with aiohttp.ClientSession() as session:
        endpoint_urls = [
            "https://example.com/api/endpoint-1",
            "https://example.com/api/endpoint-2"
        ]
        for url in endpoint_urls:
            # Check the status of the endpoint
            async with session.get(url + "/status") as response:
                if response.status == 200:
                    status_data = await response.json()
                    status = status_data.get("status")
                    if status in ["RUNNING", "INITIALIZING"]:
                        results.append({"endpoint": url, "status": status})
                        continue
            
            # If not running, send a request to start the endpoint
            async with session.post(url + "/start") as start_response:
                if start_response.status == 200:
                    start_data = await start_response.json()
                    results.append(start_data)
    
    return Output(value=results)
```  
   
  The Error:
  
  ```
  dagster._core.errors.DagsterExecutionStepExecutionError: Error occurred while executing op "start_tag_endpoints":
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/execute_plan.py", line 282, in dagster_event_sequence_for_step
    for step_event in check.generator(step_events):
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/execute_step.py", line 494, in core_dagster_event_sequence_for_step
    for user_event in _step_output_error_checked_user_event_sequence(
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/execute_step.py", line 183, in _step_output_error_checked_user_event_sequence
    for user_event in user_event_sequence:
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/execute_step.py", line 88, in _process_asset_results_to_events
    for user_event in user_event_sequence:
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/compute.py", line 198, in execute_core_compute
    for step_output in _yield_compute_results(step_context, inputs, compute_fn, compute_context):
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/compute.py", line 167, in _yield_compute_results
    for event in iterate_with_context(
  File "/usr/local/lib/python3.11/site-packages/dagster/_utils/__init__.py", line 466, in iterate_with_context
    with context_fn():
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/utils.py", line 84, in op_execution_error_boundary
    raise error_cls(
The above exception was caused by the following exception:
ValueError: <Token var=<ContextVar name='_warnings_on' default=True at 0xffff9607fec0> at 0xffff00702fc0> was created in a different Context
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/utils.py", line 54, in op_execution_error_boundary
    yield
  File "/usr/local/lib/python3.11/site-packages/dagster/_utils/__init__.py", line 468, in iterate_with_context
    next_output = next(iterator)
                  ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/compute.py", line 134, in gen_from_async_gen
    yield loop.run_until_complete(async_gen.__anext__())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/compute_generator.py", line 103, in _coerce_async_op_to_async_gen
    for event in validate_and_coerce_op_result_to_iterator(result, context, output_defs):
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/compute_generator.py", line 353, in validate_and_coerce_op_result_to_iterator
    with disable_dagster_warnings():
  File "/usr/local/lib/python3.11/contextlib.py", line 144, in __exit__
    next(self.gen)
  File "/usr/local/lib/python3.11/site-packages/dagster/_utils/warnings.py", line 152, in disable_dagster_warnings
    _warnings_on.reset(token)
  ```
  
  
I experienced this issue also in dagster==1.7.13, specifically when working with `aiohttp` and nested async calls. I've pinned to 1.7.7 temporarily until the issue is fixed in a release.
The PR that fixes this will be part of this week's release (1.7.14) usually we release on thursday afternoon



Patch:
diff --git a/python_modules/dagster/dagster/_core/execution/plan/compute_generator.py b/python_modules/dagster/dagster/_core/execution/plan/compute_generator.py
index 2b41e694304c7..3c3fd5940f8f9 100644
--- a/python_modules/dagster/dagster/_core/execution/plan/compute_generator.py
+++ b/python_modules/dagster/dagster/_core/execution/plan/compute_generator.py
@@ -351,13 +351,16 @@ def validate_and_coerce_op_result_to_iterator(
                 _check_output_object_name(element, output_def, position)
 
                 with disable_dagster_warnings():
-                    yield Output(
+                    output = Output(
                         output_name=output_def.name,
                         value=element.value,
                         metadata=element.metadata,
                         data_version=element.data_version,
                         tags=element.tags,
                     )
+
+                yield output
+
             else:
                 # If annotation indicates a generic output annotation, and an
                 # output object was not received, throw an error.


Test patch:
diff --git a/python_modules/dagster/dagster_tests/execution_tests/misc_execution_tests/test_async.py b/python_modules/dagster/dagster_tests/execution_tests/misc_execution_tests/test_async.py
index 82d4eb15698c5..1417b07ca8cb5 100644
--- a/python_modules/dagster/dagster_tests/execution_tests/misc_execution_tests/test_async.py
+++ b/python_modules/dagster/dagster_tests/execution_tests/misc_execution_tests/test_async.py
@@ -1,6 +1,6 @@
 import asyncio
 
-from dagster import Output
+from dagster import Output, asset, materialize
 from dagster._core.definitions.decorators import op
 from dagster._utils.test import wrap_op_in_graph_and_execute
 
@@ -15,6 +15,16 @@ async def aio_op(_):
     assert result.output_value() == "done"
 
 
+def test_aio_asset():
+    @asset
+    async def aio_asset(_):
+        await asyncio.sleep(0.01)
+        return Output("done")
+
+    result = materialize([aio_asset])
+    assert result.success
+
+
 def test_aio_gen_op():
     @op
     async def aio_gen(_):